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ABSTRACT 

In  the  summers  of'  1967  through  ,1970,  summer 
institutes  were  held  at  the  University  of  New  Hampshire,  Durham  to 
train/those  involved  in  the  field  of  higher  education  with  various 
research  methods.  Although  the  initial  institute  participants 
consisted  mainly  of  classroom  personnel  from  New  Hampshire,  the 
participants  in  the  three  subsequent  programs  included  educational  . 
administrators,  area  specialists  and  project  directors  who  were 
actively  involved  in  a  variety  of  Federal,  regional,  state  and  local 
educational  organization  efforts.  This  document  focuses  on  two  ma jor 
evaluative  targets:    (1)   an  evaluation  of  the  1970  summer  institute 
efforts,  and  (2)   a  longitudinal-based  assessment  of  the  instructional 
effectiveness  of  the  1967-70  institute  programs  separately  and  in  the 
aggregate.  The  results  of  these  assessments  include  not  only 
substantive  data,  but  also  their  associated  recommendations, 
(Author/HS) 
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In  order  to  keep  the  original  report  in  a  format  that  is  easily 
manageable  an  appendix  of  all  instructional  and  evaluative  materials  was 
deleted  from  the  original  report  appendix.     However,,  the  authors  of  this 
document  feel  that  it  is  important  for  parties  interested  in  viewing  the 
instructional  and  evaluative  materials  to  have  these  resources  available. 
Therefore,  a  supplementary  appendix  to  the  original  report  was  prepared. 
Six  of  these  copies  are  available >  :^ive  at  the  National  Center  for  Research 
and  Development,  Office  of  Education,  Washington,  D.C.,  and  one  at  The 
Bureau  of  Educational  Research  and  Testing  Services,  University  of  New 
Hampshire,  Durham,  New  Hampshire. 

In  the  crse  of  bulky  or  hardbound  materials  such  as.  books,  that 
■we  omitted  from  the  appendix,  arrangements  can  be  made  to  obtain  them  from 
the  publisher  or  in  some  cases,  they  can  be  obtained  upon  request  from 
the  Bureau  of  Educational  Research  and  Testing  Services,  Box  Q,  Durham, 
New  Hampshire, 

Maurice  Olivier 
Everett  Barnes 
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Dr.  Daniel  Stuff lebeam,  in  a  program  mission  statement  developed  for 
OMo  Stale  UniverrJiy,  makes  the  following  statements: 

''The  tasks  of  educotionol  developr^ienl  include  efforts  to 
understond  the  dynamics  of  educational  change,  to  identify 
targets  and  criteria  for  improvement  of  educational  systems, 
and  to  formulate,  test,  and  integrate  into  the  normal  opera- 
tion of  systems  improved  programs  or  ways  of  operating 
which  produce  higher  levels  of  systems  performance  .  .  . 
Increasingly,  however,  educational  institutions  are  seeking 
highly  qualified  professionals  in  development  specialties; 
persons  with  broad  understanding  of  the  operations  o*f  the 
education  system  with  the  ability  to  understand  global 
problems  of  large  systems,  such  as  local  school  districts  and 
state  or  national  educational  systems;  and  with  tlie  capacity 
to  organize  and  operate  staff  activities  in  evaluation,  plan- 
ning, program  management  or  research  and  development. 
These  are  newly  emprging  roles,  not  yet  clearly  or  consist- 
ently defined,  which  represent  efforts  by  educational  systems 
to  institutionalize  mechanisms  for  orderly .  systems  develop- 
ment." 

It  is  to  the  training  of  these  kinds  of  people  that  this  Institute 
directly  addresses  itself. 


THE  INSTITUTE 


The  sumin-^r  institute,  Planning  for  Educational  Decision-making 
Jn  the  70's,  sponsored  by  the  United  States  Department  of  hlealth, 
Education  and  Welfare  will  be  held  at  the  University  of  New  hlamp> 
shire  for  a  six  week  period,  from  July  6  to  August  14,  1970. 

Thirty-two  participants  comprised  of  doctoral  and  post-doctoral 
candidates  from  throughout  the  United  States  and  its  territories  will 
participate  in  an  intensive,  comprehensive  program  of  study  and 
training  in  the  methodology  and  application  of  educational  research. 

The  Institute  will  concern  itself  with  , the  problems  of  training 
educators  ot  the  doctoral  and  post-doctoral  level  in  the*  skills  of 
project  management,  educational  research,  and  evaluation— as  aids 
in  the  process  of  educational  decision-making. 

The  major  themes  of  the  Institute  are: 

1.  The  study  of  the  problems  inherent  in  the  management  of 
an  educational  evaluation  project, 

2.  The  study  of  the  problems  inherent  in  evaluating  the  par- 
ticular curriculum  changes  with  which  the  trainee  is  concerned. 

3.  The  study  of  the  major  alternatives  open  to  the  educator  in 
terms  of  educational  research  methodologies. 

4.  The  study  of  the  problems  associated  with  the  development, 
dissemination,  and  adoption  process  in  the  area  of  educa- 
tional evaluation. 

5.  The  study  of  communications  techniques  applicable  to  proper 
implementation  of  the  decision-making  process  at;  various 
levels  of  the  educational  system. 

6.  The  use  of  modern  data  processing  equipment  to  facilitate 
the  encoding  and  utilization  of  research  data. 

7.  The  reading  of  current  educational  literature  relevant  to  the 
research  project  with  which  the  trainee  is  involved. 


Program  and  Consultants 

Participating  in  the  Institute  will  be  the  following  personnel: 
Director:  Albert  Eiwell,  Bureau  of  Educational  Research  and  Testing 
Services,  University  of  New  Hampshire. 

Consultants-.  Desmond  Cook,  Educational  Program  Management 
Center,  Ohio  State  University,  a:!d  Gregory  Trezebrotowski,  Ol-.io  State 
University,  who  will  lead  the  sessions  concerned  with  project  manage- 
ment. 

Henry  Wolbesser,  University  of  Maryland,  who  v/iil  direct  the 
study  of  the  construction  of  behavioral  objectives  and  the  design  of 
assessment  tasks,  the  construction  of  learning  sequences,  anc  a  study  of 
their  empirical  foundations. 

The  third  section  of  the  Institute  will  focus  on  the  construction  of 
behavioral  objectives.  It  will  be  instructed  by  Dr,  Wolbesser  with 
the  assistance  of  Leonard  Cohen,  Educational  Testing  Service,  Prince- 
ton, New  Jersey;  and  William  Gray,  of  the  Maryland  State  Depart- 
ment of  Education. 

The  study  of  different  models  to  evaluate  education  will  he  di- 
rected by  Daniel  Stufflebeam,  Evaluation  Center,  Ohio  State  Univer- 
sity; and  C^-on  Guba,  Associate  Dean,  School  of  Education,  Indiana 
University. 

Dr.  Stuff lebeam  will  conduct  a  series  of  seminars  concerned  with 
problematic  simulation,  with  the  assistance  of  Robert  Hammond,  Ohio 
State  University,  and  Dr.  Cuba. 

The  final  sessions  of  the  Institute  will  be  devoted  to  an  intensive 
discussion  of  the  problems  associated  with  the  actual  development, 
dissemination,  and  adoption  of  educational,  innovations,  directed  by 
William  Asher/  Purdue  University,  and  Daniel  Helsey,  Whittemore  - 
School  of  Business  and  Economics,  University  of  New  Hampshire. 

Sarah  Curwood,  Chairman  of  the  Department  of  Sociology  at 
Knoxville  College  will  act  as  a  special  consultant  to  Southern  black 
educators  in  on  introductory  week  of  seminars. 


HEW  Institute 
Planning  for  Educational  Decision  Making  in 
the  70's 
Application  Form 

(detach  and  maiT before  June  1,  1970) 


name 


address 


zip  code 

home  phone  

age  u.s.  citizen? 


number  of  dependents  (excluding  yourself)  who  were  i 
claimed  on  your  last  federal  income  tax  return  (you  I 
may  not  claim  any  dependents  if  you  filed  a  joint 
return  and  were  not  the  major  wage  earner) 


are  you  presently  employed  in  a  school,  system,  or 
college  and/or  employed  by  title  I  or  title  III? 

yes  '  no   


ERIC 


1 


(if  not  presently  employed  in  a  school,  system,  or  col- 
lege and/or  employed  by  title  I  or  title  III,  omit  the 
items  on  this  page,  go  on  to  next) 


school  name 
address 


zip  code 

school  phone 

type  of  school  (technical  institute,  elementary,  junior 
high,  etc.) 

number  of  students  enrolled 


name  of  supervisor  I 

your  position  I 

■  .  ■  .'I 

.  . ...  .       _  j 

if  you  are  preparing  for  a  different  assignment,  I 

specify  here  I 


ERIC 


summarize  your  years  of  teaching  experience  and 
related  work: 

subjects  or  assignments  level  years 


list  your  places  of  employment  in  teaching  or  related 
work  in  the  past.five  years: 

place  address 


what  colleges  or  universities  have  you  attended  (most 
recent  first): 

date  institution  degree  major 


list  ail  undergraduate  and  graduate  courses  in  eval- 
uation and  measurement: 


what  teaching  certificates  or  other  credentials  do  you 
hold? 


describe  any  other  significant  academic  experiences 
you  have  had  in  the  subject  field  of  this  summer 

institute 


list  the  professional  organizations  in  which  you  hold 
membership 


are  you  applying  for  any  institutes  or  fellowships  in 
addition  to  this  one? 

if  yes,  specify  ■ '    .    ■  ■    '  ■  ■■■  •    ■  ;     '  ^.  , 

I  certify  that  the  statements  made  by  me  in  this  appli- 
cation are  true,  complete,  and  correct  to  the  best  of 
my  knowledge  and  belief,  and  are  made  in  good 
faith. 


date 


signature 


Academic  Credits 


Six  hours  of.  graduate  credit  will  be  awarded  for  the  successful 
completion  of  the  Institute.  The  participants  may  apply  these  credits 
at  the  doctoral  or  post  doctoral  level-Credits  will  be  awarded  ac- 
cording to  the  following  prospectus: 

Education  881,  (4  credits).  Methods  and  Techniques  of  Educational 
Research.  This  course  is  a  critical  study  of  the  ^principal  methods  em- 
ployed in  the  Investigation  of  educational  problems  and  an  evaluation 
of  the  procedures  and  standards  used  in  reporting  the  findings;  de- 
signed as  an  advanced  course  for  candidates  for  graduate  degrees. 

Education  895,  (2  credits).  Research  Problems  in  Education  is  con- 
cerned with  the  individual  investigation  of  a  problem  in  the  area  of 
educational  research.  This  course  will  be  used  first  to  expose  the 
trainees  to  a  variety  of  educational  research  efforts.  In  the  latter  por- 
tion, each  trainee  will  be  expected  to  develop  a  research  proposal 
which  will  hove  practical  importance  and  relevance  to  his  home  district, 
or  to  evaluate  a  project  presently  being  studied. 


Stipends  and  Fees 

Tuition  and  fees  for  participating  members  of  the  Institute  are 
provided  for  under  the  accords  of  "the  HEW  grant  for  the  summer 
program. 

Following  acceptance,  participants  may  receive  a  stipend  of 
$75.00  per  week  pi  us  . $15  per  vyeek  for  each  dependent  for  the  period 
of  attendance  at  the  Institute.  Stipends  will  .be  paid  bi-wee.kly/  on 
July  17,  31,  and  August  14.  A  maximum  of  2.5  dependents  will  be 
guaranteed  payment  throughout  the  six  weeks  of  the  Institute.  Addi- 
tional dependency  payment  will  be  determined  by  the  amount  of 
funding  available. 

Participants  will  also  receive  travel  expense  payments  for  one 
round  trip  between  their  home  and  the  University  of  New  Hampshire. 
Textbooks  and  supplies  normally  associated  with  graduate  programs 
will  be  purchased  by  the  enrollee. 

;  Since  the  success  of  the  Institute  will  depend  on  the  sharing  of 
ideas,  participants  will  be  housed  in  one  University  residence  hall 
and  will  have  their  meals  in  a  reserved  area  of  the  dining  hall.  Par- 
ticipants will  be  requested  to  Gvail  themselves  of  these  facilities  Mon- 
days through  Fridays,  and  will  be  responsible  for  their  costs.  The  total 
charges  for  the  six  week  period  are  as  follows: 

Single  room  and  15  meals/week  $228.00 

20  meals/week  264.00 

Double  room  and  15  meals/week  210.00 

20  meals/week  246.00 

A  private  Institute  reading  room  will  be  made  available  in  the 
residence  hall. 

Criteria  for  Admission 

Institute  applicants  must  be^  presently  working  in  an  approved 
Doctorate  program  in  the  area  of  Education  and/or  hold  an  earned 
Doctorate  in  the  area  of  Education. 

The  applicant  must  also  meet  one  or  more  of  the  following 
criteria: 

1,  He  must  be  one  of  the  personnel  In  any  State  Department  of 
Education  in  the  United  States,  having  responsibility  for 
school-related  projects,  or  ' 


2.  A  curriculum  coordinator,  such  as  an  Assistant  Supertnfendenf 
of  Schools,  who  has  been  designated  a  project  director; 

3.  A  member  .of  Senior  High/Junior  High,  or  Elementary  School 
staff  responsible  for  implementing  curriculum  study; 

4.  A  member  of  o  college  or  university  faculty  or  staff,  serving 
as  a  research  consultant  to  any  public  school  system; 

5.  A  staff  member  of  a  public  or  private  school  interested  in 
developing  research  competencies; 

6.  An  employee  of  an  educational  research  laboratory  or  a 
research  and  development  Center. 

Academic  Criteria:  The  applicant  must  possess  a  Bachelor's  degree 
from  an  accredited  institution,  evidence  of  satisfactory  scholarship, 
and  promise  of  ability  to  succeed  at  the  graduate  level.  An  under- 
graduate grade  point  average  of  2.5  (on  a  4.0  scale)  will  be 
minimum. 

Teaching  Experience:  A  minimum  of  two  years  in  public  or  private 
school  education  and  evidence  of  ability  to  provide  leadership  and 
responsibility  in  the  area  of  curriculum  research  will  be  required,  A 
letter  of  recommendation  will  be  requested  from  the  applicant's  im- 
mediate superior  indicating  the  meeting  of  these  criteria. 

The  Institute  will  operate  in  compliance  with  the  Civil  Rights  Act 
of  1964,  which  states:  "No  person  in  the  United  States  shall,  on  the 
ground  of  race,,  color,  or  national  origin,  be  excluded  from  participa- 
tion in,  be  denied  the  benefit  of,  or  be  subject  to  discrimination  under 
any  program  or  activity  receiving  Federal  financial  assistance." 

Application  Procedures 

Applicants  to  the  Institute  who  fit  the  above-mentioned  criteria 
will  be  selected  for  admission  by  a  Selection  Committee,  comprised  of 
individuals  directly  concerned  with  teacher  preparation  and  research 
in  education  at  the  University  of  New  Hampshire. 
•/  AppHcants  for  admission  to  the  Institute  must  return  the  enclosed 
Application  Form  to  the  Bureau  of  Educational  Research  and  Testing 
Services  no  later  than  June  1,  1970.  The  Confidential  Evaluation  Form, 


to  be  completed  by  the  applicant's  immediate  supervisor  must  olso  be 
returned  by  June  1 . 

Accepted  applicants  and  alternates  will  be  notified  by  June  15, 

A, letter  of  acceptance  from  the  selected  applicants  and  alternates 

must  be  forwarded  to  the  Bureau  of  Educational  Research  and  Testing 

Services  no  later  than  June  22, 

Recreational  Facilities 

The  University  of  New  Hampshire  is  located  in  the  southeastern 
part  of  the  state,  near  beaches,  lakes,  mountains,  and  resort  areas, 
On-campus  facilities  include  a  modern  student  union  building,  swim- 
ming pool,  tennis  courts,  ar.d  fully  equipped  field  house.  In  addition, 
the  University  sponsors  a  summer,  theater,'  concerts,  recitals,  films,  and 
other  events.  Within  walking  distance  are  shops,  restaurants,  and 
churches.  The  cultural  centers  of  Boston  are  less  than  70  miles  away. 

A  weekly  recreational  schedule  will  be  posted  for  Institute  mem- 
bers; group  activities  may  be  planned  according  to  interests  expressed. 

Calendar 

June  1,  1970.  Application  Form  and  Confidential 
Evaluation  Form  to  be  returned  to  the  Bureau  .of  Educational 
Research  and  Testing  Services,  University  of  New  Hampshire, 
Durham,  New  Hampshire,  03824. 

June  15,  1970.  Notification  of  acceptance  to  be  sent  to 
the  accepted  applicants  and  alternatives. 

June  22,  1970.  Letter  of  acceptance  from  the  selected 
applicants  and  alternatives  must  be  forwarded  to  the  Bureau 
of  Educational  Research  and  Testing  Services. 


BUREAU  OF  EDUCATIONAL  RESEARCH  AND  TESTING  SERVICES 

p.  O.  BOX  Q 
DURHAM,  NEW  HAMPSHIRE  03824 

UNIVERSITY  OF  NEW  HAMPSHIRE 
Department  of  Education 

June  19  70 


John  Dow 
123  Street: 

Durham,  New,  Mampshire  0  3824 
Dear  Mr.  Dow : 

It  i;s  my  pleasure  to  inform  you  that  you  have  been  selected  as  a  participant 
in  iTri),.e  1970  USOE  Title  IV  HEW  Research  Institute,  "Planning  for  Educational 
Decision-Making  in  the  70 's",  to  be  held  at  the  University  of  New  Mampshire 
during  the  period  July  6  through  August  14,  1970. 

Your  application  was  chosen  by  the  Selection  Committee  from  a  group  of  highly 
qwaliified  candidates.     Since  this  Institute  is  the  only  doctoral  and  post- 
dyctoral  program  of  its  kind  funded  under  the  sponsorship  of USOE/HEW* s 
National  Center  for  Educational  Research  and  Development  for  FY  1970-71,  .  you 
are  tO'  be  congratulated  for  your  selection  as  a  participant:  in  the  Institute.. 

If  )n23aii;  choose  to  accept  your  appointment  as  an  Institute  participant,  you  mu^sc 
confims  your  'Commitmeniit  by  telephone  or  cable  no  later  than  June  2^,  1970. 
Failure  tO'«et  the  uconfirmation  deadline  will  require  that  you  automatical!?:  be 
dropped  from  the  lislL  of  primary  Institue  participants  and  an  alternate  willl  be 
selected  from  the  List  of  alternate  candidates  . 

Additionally,  may  we  request  that  you  also  respond  by  mail  no  later  than  Jun:^ 
2.6  to  the  fallowing  require-d  administrative  items:   :  (a)  a  letter  of  formal 
acceptance- ;of  appointment  Iro  the  Institute  group,  and  (b)  submis^sion  of  the 
required  housing  f ©rms .     A  return  envelope  is  enclosed  for  your  convenience.  Botli 
of  these  items  should,  be  mailed  directly  to  the  Bureau  of  Educational  Reseas^cT  and 
Testing  Services,  to  facilitate  closure  of  our  contractual  requirements  prior  to 
your  arrivaiii  ;at  Durham  and  the  opening ' of  the  Institute  .Program.: 

Again,  congratulationis  are  in  order  on  your  selection  as  an  Institute  partijzipant . 
I  am  looking  forward  to  meeting  and  working  with  you  in  the  coming  year. 

Sincerely, 


Albert  R.  Elwell 
Institute  Director 


"'"elcphono:  862-1826 
062-ie?? 
S62  1046 


ME-/, 
Enc. 


BUREAU  OF  EDUCATIONAL  RESEARCH  AND  TESTING  SERVICES 

p.  O.  BOX  Q 
DURHAM,  NEW  HAMPSHIRE  03824 

UNIVERSITY  OF  NEW  HAMPSHIRE 
Department  of  Education 


June  19  70 


John  Dow 
123  Street 

Durham,  New  Hampshire  03824 
Dear  Mr.  Dow: 

I  am  sorry  to  inform  you  that  you  have  not  been  selected  as  an  active  participant 
in  the  19  70  USOE  Title  IV  HEW  Research  Institute,  "Planning  for  Educational 
Decision-Making  in  the  70 's",  to  be  held  at  the  University  of  New  Hampshire  during 
the  period  July  6  through  August  14,  1970. 

Your  application  was  reviewed  by  the  Selection  Committee  along  with  those  of  the 
total  group  of  highly  qualified  candidates.     Since  this  Institute  is  the  only 
doctoral  and  post-doctoral  program  of  its  kind  funded  under  the  sponsorship  of 
USOE/HEWJs  National  Center  for  Educational  Research  and  Development  for  FY  1970- 
71,  the  competition  was  extremely  keen  for  the.  thirty  participant  positions 
allocated  under  the  grant. 

Although  we  were  unable  to  offer  you  an  appointment  as  a  participant  in  the 
Institute,  you  are  to  be  congratulated  for  your  interest  in  national  educational 
research  and  development  activities.     We  anticipate  that  a  series  of  similar 
institutes  will  be  funded  for  FY  1971-72  at  which  time  I  would  hope  that  your 
continuing  interest  in  education  would  lead  to  an  application  to  one  of  the 
regional  institutes  in  the  coming  year. 

Again,  thank  you  for  your  interest  in  the  1970  HEW  Research  Institute.  I  am  sure 
that  you  understand  the  competitive  nature  of  the  grant  and  the  final  decision  of 
the  Selection  Committee  relative  to  your  application.  Best  wishes  for  the  coming 
year. 

Sincerely,  '  . 


Telephone:  862-1826 
862-1827 
862-1046 


Albert  R.  Elwell 
Institite  Director 


ARE/eg 
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GENERAL  INFORMATION  HANDBOOK 
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■    U.S.  Office  of  Education—  Title  IV 
Department  of  Health,  Education  and  Welfare 

NATIONAL  CENTER  FOR  EDUCATIONAL 
RESEARCH  AND  DEVELOPMENT, 


Pureau  of  Educational  Research 
and  Testing  Cervices 
Department  of  Education 
University  of  New  Hampshire 
Durham,  New  Hampshire  03824 


FOREWORD 

This  General  Information  Handbook  has  been  prepared  to  provide  Institute 
participants  and  staff  with  a  ready  source  of  primary  information  to 
assist  you  in  planning  your  Institute  as  well  as  recreational  activities 
and  to  imke  the  Institute  experience  pro fessionally  rewarding,  and, 
personally  pleasant. 

Please  feel  free  to  contact  the  Bureau  staff  should  you  jieed  additional 
and/ or  more  precise  information. 


REGISTRATION  INFORMATION 


DATE  AND  LOCATION  OF 
INSTITUTE  REGISTRATION 


Ir'WEDIATE  ITEMS  OF 
SIGNIFICANCE 


Participants  are  required  to  check  in 
: on  Sunday,  July  5  between  2:00  P.M.  and 
8:00  P.M.  at  Stoke  Hall. 

Notes :  Participants  cannot  be  admitted 
and  registered  before  or  after  the  time 
period  specified  above. 

If  you  do  not  plan  to  reside  in  the 
dormitory  you  Tnust,  however ,  report  to 
the  ■  Institute  registration  desk  at  the 
time  specified  above. 

At  the  time  of  registration  each 
participant  will  receive  the  following 
items : 


INITIAL  INSTITUTE  SESSION 


a.  room  and  board  allotment  check; 

b .  meal  ticket 

c.  parking  permit  (required--do  not 
invite  a  ticket  I ) ; 

d.  list  of  required  texts  for  purchase 
and  reference 

The  Institute's  instructional  program 
will  begin  on  Monday,  July  6.  The 
following  first-day  schedule  is 
suggested  for  your  convenience: 


a.  breakfast  will  be  served  from 
6:45  A.M.  to  8:00  A.M.  in  the 
Huddleston  Hall  Dining  Roomi  ^ 

b .  participants  will  report  to  Room  : 
218,  WSBE  (Whit temo re  School  of 
Business  and  Economics  commonly: 
known  as  'UVhiz-be'|)y  building  #40 

.  .    on  the  campus  map,  at  8:30  A.M. 
for  the  initial  Institute  session; 

c.  participants  may  purchase  all  required 
textbooks  from  the  University 
Bookstore  at  their  earliest  convenience. 

d.  additional  information  relative 

to  the  initial  week  of  the  Institute  ■ 
will  be  conveyed  by  Des  Gbok  and  staff. 


2. 


FISCAL  PAYMENT  AND  •  Stipends    Tuition  and  fees  for  participating 

STIPEND  SCHEDULE  members  of  the  Institute  are  provided  for 

under  the  accords  of  the  HEW  grant  for  the 

summer  program. 

Following  acceptance,  participants  may 
.;  receive  a  stipend  of  $75.00  per. week  plus 

$15.  per  week  for  each  dependent  for  the 
period  of  attendance  at  the  Institute.  A 
maximum  of  2.5  dependents  will  be  guaranteed 
payment  throughout  the  six  weeks  of  the 
Institute.    Additional  dependency  payment 
will  be  determined  by  the  amount  of  funding 
available . 

Participants  will  also:  receive  travel  expense 
payments  for  one  round  trip  between  their 
home  and  the  University  of  New  Hampshire, 
Travel  payment  will  be  made  upon  filing 
of  travel  voucher  subsequent  to  termination 
of  summer  institute  session. 

Textbooks  and  supplies  normally  associated 
with  graduate  programs  will  be  purchased  by 
the  enrollee. 

Stipends  will  be  paid  bi-weekly  on  ' 
July  17,  July  31  and  August  14  in  the 
,  following  amounts: 

July  5    (at  registration)  room  and 
board  allotment; 
,  July  17  1/3  of  remaining  stipend; 
July  31  1/3  of  remaining  stipend; 
AugUiit  14  final  1/3  of  stipend. 

Payment  of  Charges  for  Room  and  Board 
The  total ; cost  for  room  and  board  is  due 
to  the  University  on  July  5  (date  of 
registration).    A  check  in  the  amount  due 
for  room  and  bo^rd  will  automatically  be 
deducted  from  the  original  stipend  amount 
and  presented  to  each  participant  at  the  , 
time  of  Institute  registration.    The  checks 
are  to  be  immediately  endorsed  and  returned 
to  the  Institute  Secretary  representing  full 
payment  for  room  and  board  charges  for  the  ^, 
duration i of  the  Institute. 

Income  Tax  Exclusion    For  Federal  income 
tax  purposes.  Summer  Institute  subventions 
from  funds  of  the  U.S.  Office  of  Education 
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(including  stipends  and  dependency 
allowances)  are  considered  by  the  Internal 
Revenue  Service  to  constitute  scholarsiiip 
and  fellowship  grants,  and  are  excludable 
from  the  gross  income  of  the  participants 
either  wholly  or  in  part,  depending  on 
whether  the  individual  participant  is  or 
is.  not  a  candidate  for  a  degree.    The  total 
subvention  is  excludable  from  gross  income 
in  the  amount  of  $300.00  multiplied  by 
the  number  of  months  for  which  the  benef:  ts 
are  received  up  to  a  maximum  total  of 
36  months. 

NOTE :  the  above  information  is  subject  to 
changes  reflected  by  1970  IRS  Codes. 
For  more  detailed  information,  see  Revenue 
Ruling. 58-498,  IRS  Bulletin  No.  1958-41, 
October  13,  1958,  pp.  15-17,  on  file  in 
IRS  Offices;  or  consult  your  local  IRS. 
Office  or  tax  consultant. 


GENERAL  INSTITUTE  PROGRAM 


The  Institute's  formal  instructional 
sessions  will  be  held  daily,  Monday  • 
through  Friday,  from  8:30  A.M.  to 
4:00  P.M.  for  the  duration  of  the 
six-week  program.    On  several  days 
inforiTial  afternoon  sessions  will  .be 
held  to  provide  participants  with  the 
opportunity  to  discuss  individual 
needs  and  projects,  to  seek  consultative 
assistance  in  problems  associated  with 
computer  programming  or  statistics,  or 
to  allow  for  informal  group  or 
individual  activities  associated  with 
the  on-going  Institute  prograra. 

The  two  formal  graduate  credit  courses 
being  offered  in  the  Institute  are 
described  below: 

Education  881 — Methods  and  Techniques  of 
Educational  Research.    This  course  is  a 
critical  study  of  the  principal  methods 
employed  in  the  investigation  of 
educational  prob lems  and  an  evaluation 
of  the  procedures  and  standards  used  in 
reporting  the  findings;  designed  as  an 
advanced  graduate  course  for  candidates 
for  graduate  degrees.    (4  credits) 

Education  882 — Research  Problems  in  Education • 
This  course  is  concerned  with  the  individual^ 
investigation  of  a  problem  in  the  area  of 
educational  research.    It  will  be  used  first 
to  expose  the  graduate  candidates  to  a 
variety  of  educational  research  efforts 
reported  in  the  literature.    In  the  latter 
segment  of  the  program^  each  candidate  will 
be  expected  to  develop  a  research,  proposal 
which  wilt  have  practical  importance  and 
relevance  to  his  local  agency^  institution 
or  school^  or  to  evaluate  a  project  presently 
being  studied.     (2  credits) 


CONSULTANTS  IN  COMPUTER 
PRXRAMMING  AND  STATISTICS 


REQUIRED  TEXTS 


SUPPLEMENTARY  TEXTS 
AND  REFERENCE  MATERIALS 


A  Research  Prograiraner  and  Statistician 
on  the  Institute  staff  at  the  Bureau 
of  Educational  Research  and  Testing 
Services  will  be  available  to  all 
participants  for  consultative  assistance 
throughout  the  duration  of  the  Institute 
program- 

A  list  of  required  texts  for  the 
Institute  program  will  be  distributed 
at  the  time  of  registration.  All 
participants  are  expected  to  purchase 
these  texts  at  the  University  Bookstore. 

A  list  of  supplementary  texts  and 
reference  materials  will  also  be 
distributed  at  the  time  of  registration. 
Several  copies  of  these  materials  will 
be  maintained  at  the  University  Library, 
the  Institute  library  or  the  resource 
library  at  the  Bureau  of  Educational 
Research  and  Testing  Services,  Additional 
reference  materials  may  be  obtained  from 
several  area  sources  such  as  the  Department 
of  Education  at  the  University,  etc. 
Should  you  require  assistance  in  obtaining 
needed  materials  please  contact  an 
Institute  staff  member.    Library  hours 
are  noted  under  the  "Instructional 
Facilities"  section  of  this  Handbook. 


IMPORTANT:  P.D.K. 
MANUSCRIPT 


Manuscript  copies  of  the  P.D.K.  text 
are  being  made  available  for  instructional 
^usuage  at  the  Institute.    This  text  is 
being  made  available  through  the  courtesy 
of  P.  D.K.  and  Dr.  Stufflebeami  et  al. 


PLEASE  NOTE 


PLEASE  --ALL  COPIES  OF  THE  P-D.K. 
MANUS  CRI PT • MUST  BE  RETURNED  TO  DAN 
STUFFLEBEAM  AT  THE  CONCLUSION  OF  THE 
FIFTH-WEEK  SEGMENT  OF  THE  INSTITUTE. 
WE  CAUTION  YOU  NOT  TO  COPY  ANY  PORTION 
OF  THIS  TEXT  WITHOUT  THE  EXPRESSED  CONSENT 
OF  DR.  STUFFLEBEAM.    YOUR  COOPERATION 
IS  CORDIALLY  REQUESTED!!!!!! 


PRINCIPAL  INSTITUTE 
CONSULTANTS  AND  CALENDAR 


The  following  Institute  calendar  noted 
the  primary  consultant  and  schediiiled 
instructional  context,  OccasionaiJ^^., 
there  will  be  additional  guesr^  leclnarers 
who  will  develop  in  detail  aii...di.sciission 
and  analysis  of  a  research  aactxvity  of 
cursrent  concern.    The  principal  tostitute 
cozusaaltants  are: 


July  6 


10 


July  13  -  24 


July  27  - 
August  7 


August  10  -  14 


Drs.  Desmond  Cook  md 
Gregory  Trezebratowski — ^ 
project  management  models 
and  techniques  * 

Dr.  Henry  Walbesser  and 
staff  —  behavioral  object 
design  of  assessment  tasks 
and  construction  of  l^-r-ni. 
sequences ; 

Drs .  Dani e I  Stufflebeam^ 
Egon  Giiba  and  Robert 
Hammond  —evaluation  model 
and  techniques^  seminai*s  o- 
problematic  simulations;  a 

Drs.  William  Asher  and 
Daniel  Heisey  —  problems 
associated  with  developing^ 
disseminating  and  adopting 
educational  innovations. 


ERIC 
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UNIVERSITy  SERVICES 
AND  BECREASIOnAL  FACILITIES 


UNIVERSWY  LIBRARY 


The  University  Dimond  Library  contains 
500,000  volumes,  over  4,000  periodicals, 
a  microfilm  collection,  and  listening 
rooms  for  the  record  and  tape  collections. 
Newly  expanded  and  renovated  including  air 
conditioning,  the  library  provides  pleasant 
and  colorful  study  facilities  and  service 
areas . 


UNIVERSITY 
BOOKSTORE 


The  University  Bookstore  carries  all  text 
books,  related  reading  material,  and  class- 
room supplies  which  Institute  participants 
may  need.     The  Bookstore  also  carries  a 
large  selection  of  paperbacks  of  general 
as  well  as  curricular  interest.  Bookstore 
hours:    Monday  through  Friday j  8:00  A.M.  to 
4:30  P.M. 


HEALTH  SERVICE 


University  Health  Service,  located  in  Hood 
House,  includes  a  26-bed  hospital  and  a 
well-equipped  out-patient  clinic  for  the 
diagnosis  and  treatment  of  anibulatory 
patients.    Registered  nurses  are  on  duty 
2M-hours  a  day ;  University  physicians  and. 
the  University  ambulance  are  always  on  call 
during  the  Summer  Session,    For  ambulance 
service,  dial  (86)2-1425;  Hood  House ^  dial 
(86)2-1530.    If  off-oampus^  dal  868-5531 
for  both  services. 


MEMORIAL  UNION 
BUILDING 


The  Memorial  Union  is  a  community  center 
for  students  and  faculty.    With  its 
extensive  and  well-planned  facilities ,  it 
serves  as  a  focal  point  for  extra- 
curricular activities.    In  addition  to 
meeting  and  conference  rooms,  the  Union 
provides  lounges,  music-listening  and 
television  rooms ,  a  cafeteria  and  snack-bar 
service,  and  a  game  area  for  table  tennis , 
billiards ,• and  bowling.    The  Memorial  Union 
reception  desk  serves  as  an  information 
center  and  is  open  daily  from  8:00  A.M. 
through  10:00  Reception  desk 

telephone/ (86)2-1527. 


This  facility  includes  exhibits  by  artists 
in  residence  and  visiting  exhibitions  am"^ 
.artists  are  sponsored  regularly.  Schedules 
exhibits  are  announced  in  weekly  bulletin  . 
The  Cent€!r  is  the  home  of  the  Johnson 
Theater  \ihich  offers  regularly  scheduled 
performaHces  during  the  Summer  Session. 


This  facility  is  open  during  the  f ollov  Lng 
hours:    Monday  through  Friday  —  9:00  A  M. 
to  12  noon  and  2:00  P.M.  to  5:30  P.M. 
Saturday —  9:00  A.M.  to  12  noon  and  2:i9  P.M. 
5:00  P.M. 

Sundaij  —  P::00  P.M.  to  5:00  P.M. 

The  pool  includes  swimming  and  diving  a* 
as  well  :as  a  wading  area  for  young  chilcre^.. 
Lifeguards  are  on  duty  throughout  the  o^er 
hours . 

Admission  to  the  swimming  pool  is  based 
upon  three  types  of  privileges: 

a.  Summer  Session  -  denoted  by  special 
ticket.    Each  session  student  is  all i 
one  ticket  for  his  own  use.  These 
tickets  may  be  obtained  at  the  pool 
upon  presentation  of  a  Summer  Session 
identification  card.     Rate  for  the 
6-week  institute  is  $6*.  Any 
participant  wishing  pool  privileges 

.  for  his  immediate  family  should 
subscribe  to  the  season  family  privileges 
as  outlined  below. 

b.  Family  -  Season  tickets  may  be  obtained 
for  each  member  of  one *s  immediate 
family  (i.e. 5  husband,  wife,  dependent 
sons  and  daughters)  at  3  family  rate 

of  $25.    Tickets  may  be  purchased  at 

the  pool  office  with  proper  identification. 

<?•    Guest  -  guests  of  any  person  holding 
a  regular  privilege  ticket  may  purchase 
z  dally  pool  ticket.    This  will  entitle  ' 
the  holder  to  pool  use  for  one  day  only. 
Cost  of  the  ticket  is  50<:  and  are 
available  at  the  pool  :6ff ice .  (Note: 
should  you  own.  a  regulM*  ticket  but  not 
in  your  possession,  ^om  may  purchase 
a  daily  ticket  as  abjove.) 


TENNIS  COURTS  Several  University  tennis  courts  are 

available  for  open  use.  Special 
lighting  makes  night  tennis  possible 
on  some  courts.    The  courts  are  avail- 
able free  of  charge. 


LUNDHOLM  GYMNASIUM  This  facility  includes  indoor  swimming 

AND  FIELD  HOUSE  pool,  handball  courts,  basketball 

courts,  etc.    Nomiinal  fees  are  charged 
dependant  upon  facility  being  used. 
Outdoor  f aci -ities  include  a  track 
and  other  re'ireational  facilities. 
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PRINCIPAL  INSTITUTE  INSTRUCTIOIUL 
AND  SUPPORTIVE  FACILITIES 


PRINCIPAL  CLASSROOM  BUILDING 


Several  lecture:  and  informal  discussion 
rooms  of  the  new  Whittemore  School  of 
Business  and  Economics  Building  (WSBE) 
will  be  utilized  for  instructional 
presentations.    Flexible  usage  of  these 
facilities  have  been  obtained  for  the 
Institute  including  access  to  the  ^'Hex" 
roQm  for  informal  group  discussions. 


COMPUTATION  CENTER 


The  Computation  Center  located  in  Kingsbury 
f^ali'  houses  an  IBM  360-50  and  all  necessary 
suppox'tivia  data  processing  configurations 
including  remote  ursG^f?.  terminals.  Ti 
participants  should  contac  P.'^^^vd 
Burrows^  Manager  at  (86)2-^.323  for 
formal  introduction  to  the  facilities  o-  ua 
Center.     Access  to  the  Center  for  data 
analysis  and  processing  requirements  should 
be  coordinated  Don  Bailey  of  the  Institute 
staff. 


LIBRARY  FACILITIES 


In  addition  to  the  University  Dimond 
Library  described  earlier,  several  supple- 
mentary library  and  reference  facilities  will 
be  maintained  for  participant  usage .  These 
facilities  include  an  Institute  library 
housed  in  the  . liying  unit ,  ther  reference 
library  of  the  Bureau  of  Educational  Researf  . 
and  Testing  Services,  and  the  library  res 
center  of  the  Department  of  Education. 
Specialized  materials  will  be  made 
available  in  several  relevant  curricular 
areas  --such  as  the  Reading  Clinic  — 
and  academic  departments  may  be  contacted 
for  additional  professional  materials. 


AUDIO--VISUAL  CENTER 


The  Audio-Visual  Center J  located  on  the  firs' 
floor  of  Hewitt  Hallj  is  open  daily  from 
8:00  A.M.  to  if:30  P.M.    The  Center  is  ^eq^^V 
with  projection  equipment  and  workroor.j. 
Contact  John  Bavdwelly  Director^  at  (86) 2-2P  W 
for  further  assistance ^^^^  inquiry. 


T 
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The  television  i  tudios  are  located  in  the 
lower  level  of  t  'e  Memorial  Union  Building. 
This  educational  television  station  nas 
been  actively  in\ 3lved  in  developing  and 
programming  regie  lal  in-service  progr  mrs 
fo'.'  teachers  undf  '  EREA  Titles  I  and  ^11, 
ano  is  ':^quipp'3d  v:  .th  closed  circuit 
capabilities  and  in  inventory  of  films. 
\Hlliom  Brady  at  {  '6)2-104?  is  Mrecvor 
of  .'instructional  Stf vices. 


BUREAU  OF  EDUCATIONAL 
EESEARCH  AND  TESTING  SERVICES 


The  Bureau  is  open  I-  >nday  through  Friday 
froT    8:00  A.M.  "co  4:  0  P.M.  and  is  located 
in  1  le  Verrette  Houst       In  addition  to 
the    reference  library  noted  previously, 
the    urcau  mainiiaias    xtensive  data. 
proc  -ssing,  optical  S'  anning  and  document 
repr  -duction  fecilitie  i. 

Prin  ipal  Burec':.u  staff  associated  with  th2 
Inst -tute  are  :,dentifier   later  in  ':he 
Handbook.     Fee,,  free  to   contact  any  member 
of  tie  Bureau  staff  for  onsultative 
assi  tance  or    ust  for  ii  'ormal  discussions 
rela  :ive  to  pe  'sonnel  idG  s  and  projects. 
The  '.Regional  T  aining  Pre  tgct  for  New 
Hampshire  and  .'ermont  He '.dl  Start  Progr-r 
is  an  example  if  an  on-foiiig  Bureau 
project  as  is  che  develf  pment  of  specific-' 
for  the  Nevj  Haiipshire  S:'atewide  Testing 
Programs  v/hicl.  will  be    niplimented  in  Sept^ 


DEPARTMENT  OF  EDUCATION 


^The  majority  o  .  faculty  ■  ad  facilities  are 
''^''housed  :.n  '-^or*^' l.V  Hal."' .     ""he  Reading  ^linic 
and  Counseling  .-acilities  are  located  in 
Murkland  Hall.     Again,  fer  I  welcome  to  shcr^ 
your  ideas  and:  to  discuss    .urr en t  projects 
with  the  staff  of  the  Depa  tment. 


CENTSR  FOR  EDUCATIONAL 
FIEm  SERVICES 


The  Center  is  located  in.  tb/*  Verrette  House: 
and  serves  as  a  consultativ    and  clearing  • 
house  for  educational  projeccs  associated 
with  professional  employee  relations, 
informational  services legal  and 
legislative  services ,  long-range  fiscal  and 
plant  planning  and  services  to  school  board 
personnel  and  school  administrators.. 
Jason  Boynton^  Director J  may  be  contacted 
at  (86)2-1826. 
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LOCATION  OF  mjOR  ACAUEmC 
DEPARTMENTS  AND  ADMINISTRATIVE 
SERVICES 


Consult  Uniyjersity  Centrex  Directory^ 
pages  7-11  or  call  University 
Sioitchboard  (86)2-1234. 


DINING  FACILITIES 


Institute  participants  will  utilize  the 
Hudcleston  Hall  Dining  Room  located  on 
Hair  Street.    Hours  of  service  are: 


E-^eakfast  --  6:^5  A.M.  -  8:00  A.M- 
L incheon  —  11:30  A.M.  -  1:00  P.M. 
r.nner       ^:^5  P.M.  -  6:00  P.M. 


r' 
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PRINCIPAL  INSTITUTE  STAFF  AMD  SUPPORTIVE  PERSONNEL 


INSTITUTE/BUREAU  STAFF 
Albert  Elwell 
Donald  Bailey 

Everett  Barnes 

Sarah  Ciunjood 
Elaine  Gardner 
Carl  Kleiner 

Maurice  Olivier 

Robert  Zelonis 

GUEST  CONSULTANTS 
Desmond  Cook 
Gregory  Trezebratowski 
Henry  Walbesser 
Richard  Rosen 
Daniel  Stuff lebeam 
Egon  Cuba 
Robert  Hamiond 
William  Asher 
Daniel  Heisey 


''Don'' 

"Sardiz" 

''Elaine^' 

"Carl" 

"Moe" 

"Bob" 


"Des" 
"Greg" 
"Henry" 
"Dick" 
"Dan" 
"Egon" 
"Bob" 
"Bill": 
"Dan" 


Institute  Director 

Research  Programmer  and 
Con3ulta)^t 

Adtrtinistr:.oi'»)e  Assistant  and 
Technical  Staff  Assistant 

Orientation  Consultant 

Institute  Secretary  and 
Technical  Staff .  Assistant 
Coordinator^  Program  Maintenance 
and  Statistical  Consultant 

Coordinator^  Instructional 
Components  (Bureau) 

Coordinator^  Operations 
Component  (Bureau) 


Ohio  State  University 
Ohio  State  University 
University  of  Maryland 
University  of  Maryland 
Ohio  State  University 
Indiana  University 
Ohio  State  University 
IPurdu^  University ^ 
Wrviversity  of  Neu  Hampshire 


TRANSPORTATION  INFORMATION 


AND  ROUTING  SUGGESTIONS 


GEOGRAPHIC  LOCATION  OF  THE 
UNIVERSITY  OF  NB7  HAMPSHIRE 


AIR  (COMMERCIAL) 


AUTOMOBILE  ROUTING 


;.UTOMOBILE  RENTAL 


LIMOUSINE  SERVICE 


AIR  (PRIVATE) 


TRAVEL  AGENCY 


The  University  of  Nev;  Hampshire  at  Durham 
is  located  in  the  southeast  part  of  the 
state.    The  campus  iriay  be  reached  by  any 
of  the  following  modes  of  transportation: 

Logan  International  Airport Boston, 
Massachusetts  is  approximately  70  miles 
south  of  Durham  and  is  served  by  most 
major  airlines . 

Automobile,  limousine  or  automobile  rental 
service  will  be  required  to  reach  the 
University  from  Logan  Airport. 

Automobile  Routing        Logan  to  Rt.  C-1 
North  to  US  Route  1  North  into  Interstate 
95,  North  to  Portsmouth,  N.H.  Traffic  Circle; 
then  N.H.  Route  k  West  10  miles  to  Durham. 

Automobile  Rental       Hertz,  Avis  and  National 
Rental  services  are -available  at  Logan 
Airport.     Follow  sugges  ed  automobile  routings 
noted  above . 

Limousine  Service        C  S  J  Limousine  serving 
from  Logan  Airport  to  the  Seacoast  Region" 
and  Durham.     (One-way  fare:     $7.00;  round- 
trip  fare:     $14.00, ) 

Limousine  departs  from  Eastern  and 
American  Terminals  at  Logan  Airport. 
See  enclosed  schedule. 

Sky  Haven  Airport,  Rochester j  New  Hampshire.. 

3100  ft.  paved  runway,  lighted 
no  tower  or  traffic  control 
no  landi'iig  fee 

$1.00  tie-down  fee,  overnight 
no  hanger  facilities 

From  Sky  Haven  take  Route  16  south  to 
traffic  circle,  then  access  road  to 
Spaulding  Turnpike.     Turnpike  south  to 
Exit  7  (Rt.  108),  Rt.  108  into  Durham 
and  the  University. 

Richardson's,  Dover,  N.H.  ; 
Telephone:  742-6100 


ERIC 


CLIMATE  AND  APPROPRIATE  CLOTHING 


During  the  months  of  July  and  August  days  are  warm  but  the  evenings  are 
generally  cool.    At  times,  both  days  and  nights  are  hot  and  hurnid.  These 
"heat  waves"  are  apt  to  be  followed  by  rain  showers. 

Participants  are  encouraged  to  bring  clothing  appropriate  for  the  climate 
and  for  the  types  of  activities  they  may  choose  to  pursue.     For  daily  work 
sessions,  cool  comfortable  clothing  is  recommended.     Men  are  not  required 
to  wear  suits  and  ties. 


It  is  well  to  include  also  rainwear ^  sweaters,  sports  clothes,  and  clothing 
for  week-end  social  activities. 


COWUNITY  SERVICES 


AUTOMOBILE  REPAIRING  AND  SERVICE 

Cobb^s  Citgo  Service  -  Dover  Road  -  858-772M- 
Durham  Shell  Station  -  2,  Main  Street  -  86  8--2331 
Sony's  Esso  Station  -  Dover  Road  -  868-7719 
H.  R.  Haines  Company  -.2  Dover  Road  -  858-5536  (MA) 
Monroe  and  Sons  Gulf  Station  -  Dover  Road  -  868-7722 
Sunoco  Station  -  Dover  Road  -  868-7785 

BANK 

Durham  Tx^ust  Company  -  Madbury  Road  -  86  8-55 74 

Hours:  Monday  through  Thursday  -  9 :00  A.M-  to  2:00  P.M. 

Friday  -  9: 00  A. H.  to  2 : 00  P. M.  and  4:00  P.M.  to  6:00  P. M; 

BARBER  SHOPS 

Campus  Barber  Shop  -  Main  Street 
College  Barber.  Shop  -  Main  Street 
University  Barber  Shop  -  Main  Street 

BEAUTY  n^i^5 

Color  and  Curl  -  Jenkins  Court  -  868-2223 
Norman's  Hair  Stylists  -  Main  Street  -  868-2231 

'3US  SERVICE 

College  Corner  Bus  Depot  -  Main  Street  -  868-7411 
Busses  from  Boston  and  Northern  points  stop  regularly  for 
passenger  service. 

CHURCHES 

The  Community  Church  -  Main  Street  -  Rev.  Meury,  Minister 
Sunday  Services  at  9:15  A.M.  -  868-2435 

St.  George-' s  Episcopal  Church  -  Main  Street  -  Rev.  Albert  Sno\r 
Sundays  -  8:30  A.M.  -  The  Holy  Communion 

10:30  A.M.  -  Family  Service  of  Morning  Prayer  or  Holy  Communion 
Thursdays  -  9:00  A.M.  -  The  Holy  Communion 

St.  Thomas  More  (Roman  Catholic)  -  Madbury  Road  -  Rev.  Vincent  Lawless 
Sunday  Masses  -  8:00  A.M.,  10:00  A.M. ,  and  11:30  A.M. 
Saturday  -  5:00  P.M.  and  6:45  P.M. 
Weekday  Masses  -  5:00  P.M. 


DENTIST 


Dr.  Michael  S.  Bales  -  38  Main  Street  -  858-2311 
Tir.  Peter  Low  -  Ballard  Street  -  868-9621 

DRUG/ Cosme tic  supplies 

Brooks  Discount  Store  '  Mill  Road  -  868-7886 
Town  and  Campus  -  Main  Street  -  868-9661 

NOTE:  There  is  no  registered  pharmacist  in  Durham,  Prescriptions 
may  be  filled  at  Newman's  Pharmacy,  1  Silver  Street,  Dover, 
742-1563 

HOSPITAL 

Wentworth-Douglass  Hospital  -  789  Central  Avenue ,  Dover  -742--5252 
:  Hood  House  -  Main  Street  -  862-1530 

.  LAUNDRY  AND  DRY  CLEANERS 

The  College  Shop  -  Main  Street  -  868-2220 
,  .  Durham  Laundercenter  -  46  Main  Street  -  858-7735  (self-service) 
Great  Bay  Cleaners : -  Jenkins  Court  -  868-2488 
.  Wildcat  Wash  -Mill  Road 

NBVSPAPERS  AND  mGAZINES 

Town  and  Campus  -  Main  Street  -  868-9661 
Young's  Donut  Shop  -  Main  Street  -  868-2688 

PHYSICIAN 

Hood  House  Infirmary  -  Campus  -  (86)2-1530 
■  Dr.  Rudolph  Hoene  -  Mast  Road  -  868-2261  ( internal  medicine  and  allergy) 
Dover  office:  742-3664 
Dr.  Joseph  Evora  -  Madbury  Road  -  858-7595 

POST  OFFICE 

Main  Street  -  868-2151  , 

PUBS  '  •, 

The  Down  Under  -  lower;  level  College  Corner  -  86  8-7ml 
The  Keg  Room  -  lower  level  Pizza. Den  -  868-2^85 
The  Wildcat  Pizza  -Main  Street  -  858-5530 

SHOE  REPAIR 

■         Price's  -  Main  Street  -  868-7t|.%0 

Shoes  left  here  to  be  sent  to  Dover  Shoe  Hospital.    Rapid  service 


SNACK  BARS 


College  Corner  -  56  Main  Street      868-7mi  • 
.  The  Dairy  Bar  -  off  Main  Street  (old  railroad  station)  • 
Little  Horn  -  Main  Street  -  868-2300 
Pizza  Den  -  Main  Street  868-2485 
Wildcat  Pizza  -  Main  Street  -  868-5530 
Young's  -  Main  Street  -  868-2688  . 

STORES 

The  College  Shop  -  Main  Street  -  868-2220 
Perkins,  Ltd.  -  Mill  Road  -  858-5366 
Shaine's  Clothing  Store  -  Jenkins  Cp-urt 
Yankee  Drummer  -  Main  Street  -  868-2174 

Community  Market  -  Main  Street  -  868-2521 
Durham  Shop  and  Save  -  Mill  Road  -  868-2500 
Newsky's  Ideal  Food  Store  -  Madbury,  Road  -868-2121 

Durham  House  -  44  Main  Street  -  868-2583 
Price's  Record  Shop  -  Main  Street  -  868-7410 
Red  Carpet  -  Main  Street  --  868-9661  (Town  and  Campus) 

Hardware  House  -  Jenkins  Court  -  868-9601 
Home  Care  Center  -  Mill  Road  -  868-5224 


Richardson  Agency  -  600  Central  Avenue,  Dover  -  742-6100 
TYPEWRITER  RENTAL 

E.  H.  Quimby  and  Company  -  362  Central  Avenue^  Dover  -  y^a-SSlS 


Clothing  - 

Food  - 

Gifts  - 

Hardware  - 
TRAVEL  AGENCY 


COMMUNITY  AND  AREA  RECREATIONAL  FACILITIES 

BEACHES    (ocean,  all  within  thirty  miles  of  Durham) 

in  New  Hampshire 

Hampton     Route  lA  -  Hampton 

North  Hampton  -  Route  lA  -  North  Hampton 

Rye  -  Route  lA  -  Rye 

Wallis  Sands  -  Route  lA  -  Wallis  Sands 

in  Maine 

Long  Sands  and  Short  Sands  -  Route  lA  -  York,  Maine 
Ogunquit  Beach  -  Ogunquiv,  Maine 
Wells  Beach  -  Wells.,  Maine 

HISTORICAL  SITES  IN  NEW  HAMPSHIRE 

Hudson  (near  Nashua)  -  Benson's  Animal  Farm  (great  for  adults  and  childrc:.; 

Concord  -  State  Capitol  and  historical  museums 

Exeter  -  Revolutionary  capital  of  New  Hampshire;  Garrison  House  (1550-1750); 
Phillips  -  Exeter  Academy 

Farinington  -  Schulier  Museum  -  Medieval  arms  and  armor,  palace  furniture 

Manchester  -  largest  city  (pop.  80,000f);  varied  exhibitions;  shopping 
centers 

Mystery  Hill  -  North  Salem  off  Rt.  111.    Man-made  rock  sculptures; 

archeological  excavations.    Artifact  display;  picnicing. 

Mew  Castle  -  E.  of  Portsmouth  at  Piscataqua  River  -  Ft.  William  and 
Mary;  boating  and  fishing. 

Portsmouth  -  Colonial  capital  of  New  Hampshire.    Strawberry  Bank;  several 
.      historical  homes  and  buildings  open  to  public;  shops. 

(Also,  consult  Durham  Historical  Society       Town' Hall,  Main  Street  for 
additional  touring  suggestions),. 

HIKING^  FISHING.  CLIMBING  AND  CAMPING 

White  Mountain  National  Forest  (70  miles  north)  includes  Wilderness 
Highway,  several  routes  for  touring  White  Mountain  area;  trails, 
campgrounds,  and  6,2881  Mt.  Washington  in  the  Presidential  Range- 
summit  accessible  by  auto  road  ( toj.l) ,  cog  railway  or  on  foot  trails; 
■ ;  late  "Spring  Skiing^'  may  still  be  possible  at  Tuckerman's  Ravine 
(natural  glacial  bowl). 


Lakes  Region  -  Central  New  Hampshire  -  boating,  camping,  fishing, 
swimming. 
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DINING  AND  ENTERTAINMENT 

RESTAURANTS 

In  Diwhm 

New  England  Center  for  Continuing  Education  (adjacent  to  the  campus) 
excellent  cuisine;  phone  for  reservations  (86)2  -2800 

In  Newington^  N,U.  (10  minute  drive) 
The  Mariner^  Dover  Road  742-5414 
Flagstones^  Dover  Road  --436-5726 
Newick^s  (seafood),  Dover  Point  Road  742-3205 

In  Dover  J  N.H.  (15  minute  drive) 

EMard  Johns on[Sy  Central  Aveyiue  "  742-1890 
Ramada-'Sterling  Motor  Inn ^  Silver  Street  —  742-4100 
.  l'Jeeks\Ioe  Cream  Shop  md  Char- Gi^^     New  Rochester  Road  •'-742-'5056 

In  Portsmouth  J  N.H.  (MO  minute  drive) 

Hsherinan^s  Pier  (^^afood  specialty) ^  State  Str 

(phone  for  reservations) 
Holiday  Inn^  Rotary  Circle 
Howard  Johnson  [s J  Rotary  Circle 

Meadowbrook  Motor  Inny  Rotoj^y  Circle— -  436-2700  • 
yoken^s:Thar  She  Blows,  Rt.  1,  Lafa^^  436-8224 

In  New  Castle,  N.H.( 20  minute  drive) 

Wentworth-By^The-Sea,  New  Castle  (phone  for  reservations) 

In  Rye,  N.E.  (25  Mnute  drive) 

South  Wind  (Chinese^ American)),  Rt.  1,  Lafayette  Road  —  964-5545 

In  Exeter^  N.H.  ( 25  mmute  drive) 

Exeter  Inn,  90  Pwiu;'  Street      772'-50S8  (phone  for  reservations)  : 
Ku^tz  Restaurant,  Wii&er  S  ^- 
Weeks    Ice  Crem  Shc^:  and  C^ar-G^^^      Rt.  108  772-5901 

In  Hampton,  N.H.  ( 30  TiBrnte  drive) 

Bob  Taylor^ s  Homestead,  off  Hampton-Exeter  Expressway  772-4386 

(phone  for  reserm^ons) 
Lamie  's  Tavern,  Rt.       490  Lafayette  Road  --962-8911 

,  In  Rochester,  N.H.  C 30  mnute  drive) 

Cardinal  ^ s  Restmrantj  Rt.  ll  ~  332--6U0 
Colby  ^s  Res taximnt  --  332-9891 
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In  Kittery\  Maine  (ZO  minute  Birlve) 

Dragon  Seed  (Chinese- AweHam)  J  State  Road  "  ^Qt^-iZd-llOO 
Valleys  :Siteak  Hotis.e^  Intevsimte  Uighwoy 
Warren^s^job star  House 

JTji  York ^  Maine  (40  minute  drivBj 

pyjmett's  Restaurmt^  16  Lang  Sands  Road  207-363-4902 

The  Goldenrod  LuncTkeonettrmd  Fomtain^  York  Beaoh  —  207- 363-3621 

Spiller's  Restaurant^  Ocean  Avenue       207-36 3'-3903 

.Nti)ble  Light  Dinirig  Room 

YorkDoy  Restaurant^  63  U.S.  Route  1  South  207-363-4453 


The  sumner  movie  and  drive-in  theaters  in  the  area  are  listed  belou): 

In  Durham^  New  Hampshire 

.  Summer  theater     University  of  Nei)  Hampshire  -  Paul  Creative  Arts 

Center  -  information  (C6)2-'1234 
i..,  :'-^  theater  --  The  Franklin  -  Jenkins  Court  -  868-2751 

In  Dovarj  New  Hampshire 

*  ^-7-^  -t^,^.  ^  p^^,.,,^  _       rjjj^^^g^  Street  -  742-2454 

Uptown  -  4  BTCrfr^-iraj 
Drive-in  theater  -  Route  16  -  Roches' ar^Road  -  742-3559 

xn  IJ^iioington 

■Drioe-in  theater  -  Naoington  -  4Z6-2409 

.\n  Po-r-ismouth:,  New  Jiampshire 

Smmer  theater  -  Theater  by  the  Sea  -  Ceres  Street  -  431-6660 
'Movie  theaters  -  Civic  -  Chestnut  Street  -  436^5710 

Cimema  -  10  Ccngress  Street  -  436 -:2B05 

>4ovio  thecte:-"  -'  Icka.Th^ 

hlSoohesteryNc'Hampshi^ 
^mf^r  iheatc  :  -  Roches t^ 

.2Etve-in  thea      -  Roc^^est^r  Im'ibe-in  -  Milton  Road  -  332-3204  } 

laapton^  NewZE:  rpshir^.  '     .  . 

mrivrer  theater  •  Hampton  Playhc^e  -  357  Mnnacunnet^M^^ 
Mroie  theaters  -  Casino  Theater -^^^^^ 
"  .      .  Surfmieater  -  ^65  Ocean  B^^ 

Drive-in  theater     Seacoast  Drive-in  ^  Lafayette  Road 

In  Kittery^  Maine 

Drive-in  theater     Kittery-York  -  State .  Road  -  207-439-9328 
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In  Ogunquitj  Maine 
:  Simmer  theater  -  Ogunquit  Pl^         -  Route  1 

In  Arundel^  Maine 

Sicmer  theater  -  Arundel  Plaifhous:e  --  Route  1 
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"THIS  MAS  SOMETI^€  A  PARADOX, 
BUT  NOV/  THE  TIME  GIVES  IT  PROOFE." 


Hamiet 


ERIC 


Educational  Program  Managenient  Center 
r  Educational  Deve! op^nent  Facul  ty 

|;    :  . '  ,  Co!  Uge  of  Education: 

The  Ohio  State  University  . 
rr  ■  Columbus/  Ohio  ^0210 

r. 
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Planning  for  Educations  i  Decision  Making  in' the  ^70\s 

H.E.W.  inst  itute 

.  "    '  July  6'.|0,  1370 

University  of  New  Hampshire 
; Durham^  Hew  Hampvihi  re 


Col  lege  or  tauccition 
Columbus,  §hio  43210 

Session  No.  PQte  _^  ^„  Instructor 

Topic:    Hature  of  Project  Management 

A*    Pro§ec.t  Management  Defined: 

B.    Project  Characteristics: 
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A  rather  recent  facet;  iii  curriculum  construction 
theory  is   the  development  of  curricula  via  a  behavioral 
objective  learning  hierarchy  viewpoint.     It  is  stated  in 
the  research  that  curricula  constructed,  within  the  learning 
hierarchy  domain  are  meritorious,  from  pedagogical,    as  well 
as  cognitive  and  affective,    dimensions. _    But  conspicuously/ 
absent  from  the  literature  is   a  set  of  numerical  procedures 
needed,  to  test  the  validity  of  a  curriculum  constructed  as 
a  hierarchical  learning  sequence .     There  is  also  a  dearth  - 
of  numerical  procedures  needed  to 'deal  with  questions 
concerned  with  the  comparisons   of  hierarchical  learning 
structures..    Tl.erefore,   the  purpose  of  this  article  is  to 
discuss  numerical  considerations  dealing  with  the  validation 
and  comparison  of  curricula  developed  as  behaviorally  stated 
learning  hierarchies. 


LE/vJlNING  KIERARCI-JIES.--NUMERIC/a  CONSIDERATIONS 

Theodore  A.  Eisenbex-g 

:   The  developraent  of  curricula  via  a  behavioral 
objective   learning  hierarchy  viev/point   is   a  rather  recent 
facet  in  curriculum  construction  theory.     Numerous  articles 
have  been  written  describing  the  pedagogical  and  philosoph- 
ical dimensions   associated^  with  the  specification  of 
expected  behavioral  performances  as   a  result  of  instruction 
(Gagne,    1965;    Shuiman,    1968;   V7albesser,    1966).  The 
empirical  research  of  Carter    ( 1969) ,    Gagne    ( 1967 ) ,  and 
Smith    (1970):  imply  that  the  sequencing  of  behavior al 
objectives   into   a  learning  hierarchy  seems  to  be   a  wise 
developmental  tactic  for  curriculum  construction.     But,  . 
incorporation  in  the  -literature  of  numei-ical  procedures  to 
test  researchable  hypotheses  in  this  area  is  lacking. 
Hence,    it  ' is  the  intent  of  this  article  to;  discuss  numer- 
ical considerations  dealing  with  the  validation  and 
comparison  of  behaviorally  stated , learning  hierarchies . 
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Bacj-iQrounci 

.The  con;:i  tructicn  of  a   learning  hierarcl-iy  commences 
with  the   specification  of  i}\  beiia'vj.or  a\  student   should:  be 
capable  of  per f orriiir^g   as   a   result  .of  instruction.  This 
terminal  boh.avior  h^covries   the   apeM  of  what  eventually  will 
be   a  pyramid  of  prerequisite  beha^-  S-^rs  -      The  curriculum 
developer  then  asks:     vrij at  behavior  must  one  exhibit  before 
he  can  perform  the  terminal  behavior?     By   ansv/ering  this  . 
question  he  has   idenrified  a  terminal  behcivior  and  several 
Ist-order  subordinate  behaviors*   ^  For  example,    suppose  that 
in  order  to  perform  an   identified  terminal  behavior,  one 
must  first:  .be  able  to  perform  subordinate  behaviors   a  axid  Id  . 
The  pyramid  begins  as   shown  in  Figure  1. 


Prerequisite 
subordinate 
behaviors 

Figure  1 

Identification  of  Ist^-level 
Kvibordinate  Behaviors 


Terminal  behavior 


To  each  subordinate  behavior  the  question  is  repeated:  What 
behaviors  must  one  exhibit  before  he  can  perform  this  subordinate 


behavior?     Suppo-i,ci   that   m.  order   to  par  form  behavivor  ii,  one 
n^ust   first  be  able  to  perform  behaviors         d,    and  e,    and  for 
behavior  b,    one  niust.be  aJDle  to  perform  behaviors   f ,  and 
h-     Hence,    2nd-level:  subordinate  behaviors  have  been 
identified  and  tlie  hierarchy  develops   as   in  Frgure  2. 
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Figure  2 

Identification  of  2rid-level 
Su?oordinate  Behaviors 


The  procedure  is  "iterated  until  all  the  identified  pre- 
. requisite  subordinate  behaviors   are  without  instruction 
capabilities  of  the  learner. 

i  ■ 

Gagne    (1965)  ,    Lindvall '  (1964)  ,   Mager  (1962), 
Tyler    (1950),   Wallsesser    (1968 ).    and  others  have  espoused 
the  developmental  and  procedural  considerations  for  the 
construction  of  curricula  in  the  aforementioned  manner. 


content  Val idity 

Once  the  curriculurA  'hc\s  been  constructed   an  obvious 
question  to  ask  is:      Does   the  curriculura  aor.ieve  its 
objectives?     That   is,    after   instruction  can  the  student 
perform  the  behaviors   stated  in   the  cells   of  the  -learning 
hierarchy?     In -an  effort  to  an3v,^er    this    question  a 
competency  measure  is  constructed. 

Each  cell  in  the  hierarchy  is  translated  into  a  set 
of  problems  or  questions  which  the  authors  of  the  develop-- 
mental  sequence  consider  rea.sonable  for   assessing  the 
acquisition  or  nonacquisition  of  the  stated  behavior.  If, 
after  instruction,    the  student  can  perform  the  representa-  ./ 
tive  problem^s  for  a  given  cell/  then  the  behavior  in  that 
cell  has  been  realized.     But  it   is  extremely  important  that 
the  competency  measure,    used  to  ascertain  the  acquisition 
of  the  behaviors  in  the  cells  of  the  hierarchy,  possesses 
content  validity. 

Content  validity  of  the  competency  measure  for  a 
learning  hierarchy  is  determined  in  the  usual  manner.  Given 
the  list  of  behaviorally  stated  objectives  and  the   list  of 
tasks   supposedly  based  upon  the  ob j ectives ,    can  a  group  of 
"experts"  match  the  tasks  with  the  corresponding  behaviors? 
Usually  included  within  the  representative  set  of  tasks  are 


questions   re] ated  to  no  cell   in  the   learning  hierarchy  and 
questions   represent-at x^^e  of  more  th^n   one  cell   in  the 
hierarchy  V     Adopt.LOn  of  such  a  tsct  .ic  enai:;les  one  to  ^ichxeve 
a  more  accurate  assessment  of  the  coni:ent  validity  of  the 
competency  raeasure  rather   than  by  liaying   a  1-1  correspondence 
betv/een  behaviors   and  representative  tasks.     Hence,   .it"  can 
be  assumed  that"  a  comioetency  measure  possesses  content 
validity  if  there  exists  a  mfnimum  of  a  ' 0 .90  correlation  in 
the  matching  of  each  b  eh  a.  v  i  or  ally  stated  objective  and  the 
tasks  used  to  test  the  acquisition  of  the  objective. 

Wa  lb  e  s  s  e r  Val i d  a t  i  on : 
Procedure 

:Walbesser    (1963)  has  developed  a  numerical  procedure 
for  testing  a  hypothesis  of '  "task  dependency "   in  a  leax-ning 
hierarchy.     His  procedure  provides   a  nuraerical  criterion  for  - 
determining  the  validity  of  a  hierarchicaly  organized 
learning  sequence . 

consider  the  following  schematic  representation  of ■  : 
a  learning  hierarchy  as  represented  in  Figure  3.  : 
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•  Figure  3  .  . 

Schematic  Representation  of 
a  Learning  Hierarchy 

A  "terminal  behavior"   and.  itsvinriinediate  subordinate  behaviors 
constitute  a  .hypothesis _ of  tas-'k  dependency  within  the 
hierarchy.     Figure  4  represents  the  four  hypotheses  of  the,  • 
structure  in  Figure  3.  " 


Hypothesis  2 
H(2) 


Hypothesis  1; 
H(l) 


Hypothesis  3 
H  ( 3 ) 

Hypothesis  4 
H  (4) 
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Figure  4.. 

Representation  o£  the  Four  Plypotheses 
of  the  Structure  in  Figure  3 


In  other  words,   it^  is  hypotnesized  that  in  order  to  perform 
the  behavior  required  in  cell  a,   one  must  first  be  able  to ■ 
perforin  the  behaviorG  in  cells  c  and  d.     Similarly,   in  order 
to  perform  the  behavior  in  cell        one  must  first  be  able  to 
perform  the  behaviors  specified  in  cells  g,  h,   and  i. 

;    In  Figure  4,  .consider  the  hypothesis  of  task 
dependency  that  the  behaviors  ir.  cells  c  and  d  are  pre- 
requisite for  performance  of  the  behavior  in  cell  a.  We 
/are; considering  the  following  hypothesis :  I ■  ' 


Suppose  that  the  behavior  in  each  cell  has  been  translated 
into  a  set  of  valid  tasks  that  can  be  used  to  assess  the 
acquisition  or  nonacquisition  of  -the  given  behavior .  After 
instruction,  the  competency  measure  is  administered  and 
scored  on  a  binary  scale":     1  if  in  the  judgment  of  the 
instructor  the  student  has  mastered  the  behavior  in  the 
cell,   0  if  the  student  has  not  mastered  the  behavior .  The 
possible  outcomes  of  such  a  consideration  are  listed  in 

Table  l:.- 
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0  0       OM       1  0       l.^^l       C.-O       0-^1       l-'o  11 

^   ,  :  \  

Table  1 

Possible  Outcomes  Usixng  a  Binery  Scale 
to  Represent  the  Acquisition  of  the 
Behaviors  in  the  Hypothesis  (lil 

  ■     ■   - 

)     -     ■  ,  ,.  ■ 

0 

The  1  1  configuration  implies  that  the  student,  after 
instruction,  could  perforni  the  subordinate  behavior  in 
cell  c  and  cell  d,  but  could  not  perforra  the  "terminal'' 
behavior  in  cell  a.     Similar  interpretations  could  be  given 
for  the.  other  seven  configurations. 

Each  configuration  in  Table  1  can  be  considered  as  . 
an  ordered  pair  of  digits  .     The  .  first  element  in  the  ordered 
pair  is  associated  with  the  acquisition  or  nonacquisition  of 
the  terminal  behEvvior  in  the  hypotheses.     The  second  eler.ent 
in  the  ordered  pair  is  associated  with  the  acquisition  or 
nonacquisition  of  the  totality  of  all  hypothesized  pre- 
requisite subordinate  behaviors .     if  the  student ^has 
acquired  the  terminal  behavior  and  all  hypothesized  subord^ 
inate  behaviors,  then  the  ordered  pair  associated  with  that 
hypothesis  of  task  dependency  is;  (1,1)  •     However/-: i 
t  student  has  acquired  the*  terininal  behavior  of  the  hypothesis 


but  not  all  subordinate  behaviors,   then  the  ordered  pair 
associated  with  the  hypothesis-  is    (1,0).     The  ordered  pairs 
associated  with  the  configurations  in  Table  1  are  listed  in 
Table  2. 


0°1 


1% 


1°1 


0^0 


0^1 


1 

11 


Table  2 

Ordered  Pairs  Associated  with 
Configurations  in  Table  1 

The   (0,0)   ordered -pair  implies  that  the  student  did 
not,   according  to  his  performance  on  the  competency  measure, 
acquire  the  terminal  behavior  of  the  hypothesis  and  did  not 
acquire  all  subordinate  behaviors.     A  (0, 1)  ordered  pair  : 
impli-s  that  the  student  did  not  acquire  the  terminal 
behavior,  but  did  acquire  all  hypothesized  subordinate 
behaviors,     similar  interpretations  can  be  given  to  the 
(1,0)   and   (1,1)  ordered  pairs.     Using  this  tactic,  Walbesser 
states  several  implications  of  "task  dependency"  from  a 
numerical  viewpoint. 

•     consistency  Ratio.     if  after  instruction  the  student 
^■^t^^^^^^^^^  the  \terminal  behavior '  of  ?  t^ 

°    ,    then  he  should  also  have  acquired  all  subordinate  ■ 
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behaviors.     Quantitatively,   the  consistency 
implication  of  a  hypothesis  of  a  hierarchy  is 
defined  to  be  the  ratio  of  the  frequency  of . the 
(1^1)   ordered  pair  for  the  Zaypothesis  divided  by  , 
the  sura  of  the.  frequencies  of  the    (1,1)   and   (1,0)  / 
ordered  pairs  for  the  hypothesis.     This  ratio 
ranges  frcra  0  to  1  provided     that  the  frequencies 
of^the   {1,1)   and  (1,0)  ordered  pairs  are  p.ot  both 
zero.      (In  that:  case,   the  ratio  is  not  applicable 
since  v^e  would  be  dividing  by  zezo.)     S^CiiDolically ^ 
one  is  testing  the  implication  that  acquisition  of 
the  terminal  behavior  implies  acquisition  of  all 
subordinate  behaviors. 


r,'       .   V/.,  '     f  (1/1) 

\    ■  f  (1,1)  -f  f  (i,o) 


Adequacy  Ratio.  •  If  the  student  has  acquired  all 
subordinate  behaviors,  then  v/ith  appropriate      -  ■ 
instruction  he  should  be  capable  of  performing  the 
terrainal  behavior  'of  the  hypothesis.  Numerically 
the  adequacy  of  a  hypothesis  in  a  learning  hierarchy 
is  defined  to  be  the  ratio  of  the  frequency  of  the  ^ 
(1,1)  ordered  pair  for  that  hypothesis  divided  by  v 

,  :     :  .  ■■         ■    ■       ■     I     ■  ■•■  :  ,         ....  ,  :    .    .    ■  ... 

i 

the  sum  of  the  frequencies  of  the   (1, 1)   and  (0, 1)  > 


ordered  pa.rs  for  the  hypothesis.     Thxs  ratio  also 
ranges  from  0  to  1  provided  that  the  set^  of   (0, i) 
and  (1,1)  responses  is  not  erapty.     Syn^.olically , 
:  the  adequacy  hypothesis  tests  the  xmplication  that 
^  acquisition  Of  all  subordinate  behaviors  implies, 
with  instruction,   acquisition-  of  the  terminal 
behavior . 

^      (Instr . ) 

.Vsb     =^     T  f(l,l) 

f  (.1,1)"  +  f  (0,  iT 

'^^^^^^^£^^~£2ns^^  The  inverse  consistency 

ratio  tests  the  implication  that  nonacquisition  of 
the  terminal  behaVior  implies  nonacquisition  of  all 
subordinate  behaviors .     Hence,  it  is  defined  to  be ' 
the  ratio  Of  the  frequency  of  the   (0,0)  ordered  pair 
for  the  hypothesis  divided  by  the  sum  oi  ..he 
frequencies  of  the  (0,0)   and   (0]1)  ordered  pairs  '  / 
for  the  hypothesis.     Provided  that  the  set  of 
responses  for  the   (0,0>;and  (0,1)  ordered  pairs  is 
nonempty/  this  ratio  also  ranges  from  0  to  ll 
Tubolically,  it  tests  the  following  implication;^  : 


f  (0,0)  +  ,f  (0,1)  ' 
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4-     Ilivsrse_^qu£c^^  The  inverse  adequacy  ratio  • 

tests  the  implication  that  nonacquisition  of  all 
subordinate  behaviors  implies,   even  with  media  ig 

V-.ruction,   nonacquisition  of  the  -.erminal  behavior. 
.■  .erse  adequacy  is  defined  to  be  the  ratio  of  the 
frequency  of  the   (0,0)   ordered  pair  for  the  - 
hypothesis  divided  by  the  sum  of  the  frequencies:: 
Of  the   (0,0)   r,.nd   (1, 0)   ordered  pairs  for  the 
hypothesis.     Provided  that  the  frequencies  for  (0,0) 
and   (1,0)   ordered  pairs  are  not  both  zero,  this 
ratio  ranges  from  0  to  l.     Sy^ribolically ,   the  impli- 
cation being  tested  can  be  represented  as  follows: 

(Instr . ) 


f  (0/0)        f  (1,  0)  \ 

•  Comgleteness__^a^      In  an  effort  to  determine  the  • 
strength  of  a  hypothesis  of  task  dependency  a      -  . 
completeness  ratio  is  defined.     The  completeness 
ratio  for  a  hypothesis  of  task  dependency  in  the' 
hierarchy  is  defined  to  be  the-.^ratio:of  the  frequency 
of  the   (1,1)  ordered  pair  for  the  hypothesis  divided 
by  the  sum  of  the  frequencies  of  the    (1,1)   and   (0,0)  ■ 
ordered  pairs  for  the  hypothesis:  f(l,l) 

f  (1,1)  +  f  (o,oT  ■ 
This  ratio  estimates  the  percentage .of  individu  Is 
capable  of  traversing  the  hypothesis  as  opposed  to 
those  incapable  of  performing  at  least  one  of  the 
:suborQii;ate-ceTl&  in  the  "  hypothesis  .     The  'implica- 
ti.prt£;  of  such  a  ratio  are  discussed  below.  .  , 


Hence,   an  adequacy  ratio,   completeness  ratio, 
inverse  adequacy  ratio,    inverse  comioleteness  ratio,   and  a 
completeness  ratio  can  be  associated  with  each  hypothesis 
in  the  learning  hierarchy.'     A  learning  hierarchy  is  con- 
sidered;  valid  if,. and  only. .if,   an  adequacy,  consistency, 
and  completeness  ratio  of  at  least  0.85  is  obtained  for 
each  hypothesis  in  the  learning  sequence   (Walbesser,  1969). 

I  mp  1  i  c  a  t  ioi2s_ 

In  Table  3  consider  the  following  frequency  distrib-^ 
tion  for  a  hypothesis  of  a- hypothetical  learning  hierarchy. 


Outcome : 

X 

Frequency  of  #  of 
Students  Obtaining 
Outcome:     f (x) 

(0,0) 

50  ; 

(0,1) 

0 

(1..0) 

0 

(1.1) 

-Q 

Table  3 

Preguency  ijistribution  for  an  incomplete 
■  Hypqthes Is  of  a  Hypothetical 
Learning  Hierarchy 
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The  ratios  for  the  task  dependency  hypothesis  are 
computed  as  fallows:  • 


{cor..s_3tency;     -  ^     i .  oq 

(adequacy}  =     -|iL.  i.qO 


Px,^  (inve.rse 

.  50 
cciisistc-nc:y )   =   =:  1.00 


5u-f  0 


R .  (inverse 

4 


adequacy)         =    --r..£_y^  1_00 
Pv^.  (completeness)  =    ~.      =  .50 

Even  though  the  consistency  and ' adequacy  ratios  are  1.00, 
the  data  suggest  that  50%  of  the  population  could  riot 
perform -any  of  the  behaviors  required  in  the  hypothesis  of 
task  dependency.     Hence,  to  the  curriculum  developer,  this 
suggests  that  thr.  hierarchy  should  be  extended  further,  to 
build  upon  the  behaviors  the  students  do  possess.-  Based 
upon  the  data,   one  can  not  reject  the -hypothesis  of  task 
dependency,   for  if        individual  could  perform  the  subord-  ' 
Inate  behaviors,  with  instruction,  he  can  successfully  .  : 
perform  the'  teriiiinal-  behavior  'in  the  hypothesis .     But      .  ■ 
psychologically- "a  learn ing  h ier archf  that ■ is  applicable  to 
only  50%  of  the  population  that  will  use  it  leaves  something 
.toibe  desired.   ^  Hence,  even  though  acquisition  of  :the " 
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subord  inate  behavj  o^*<^    i-'-^l         n.-^rM  -  .  r 

J^qj..    Lt.\on  oi  tho  terminal 

behavior,    one     c- 1 '-^r^-h.^  >   - -.^-*h,^o  ;  c.    - i  .-^  ^ 

;    '   'HO::..:,      as  rK-:iny  valid  for  the  • 

entire  populnt j on . 

if  a  hypothosi?   lo  cons.suoiLt  o.;L  not  adequate  ti.is 

implies  th^ifc  possihl^'  f  u^>.>-ps    >  c  -4- 

''^^'-■^-■^.^  Oil  a-  jump  between 

the  subordinate  b-^^^^-^.i^-' nr-  ^  hr.       v-,^-      i  -    .      ■  ^ 

hypothesis.     in  oiht-^^  v\->^-r;^     ;i  r-.-o- ^    ^         ^    -  ■ 
hypothesis  implies  mediating  celis  ne-d  to  be  added.  The 
frequency  distribution  given  in  Table  4  depicts  a  hypothesis 
that,  is  consistent  but  not  adeouate. 


f  (x) 

(0,0) 

0 

(0,1) 

(50) 

(1,0) 

0 

(1,1) 

(50) 

needed 
cells 


- ,    :   TablG:  4 

.Frequency  Distribution  and  Schematic  Representation 
of  an  Inadequate  but  Consistent  Hypothesis 
■J^-^      Kj'-oothetical  Learning. ;HierarchK^^^  I, -  - 


J  If  a  hypothesis  is  adequate'  but  not  consistent ,  then  one  ; 
■should  re-examine  the  subordinate  cells  to  ascertain  whether 


or  not  they  are  really  necessary  for  acquisition  of  the 
behavior  in  the  tL:^nuinaI  c::r::],.l  of  the  hypothesis. 

Tiie  acGquc'Cy  iritxo  cincl    i  nverse  cons  is  toncy  ratio  are 
not  loq-Lcally  oquivaleirt,  tiie  auoquacy  ratio  implies  the 

use  of  mediatinq  msf t ruct icri .     Moreover,    it  can  be  argued 
that  the  inverse  coiisiscenc^/  and  inverse  adequacy  ratios 
are  'Useful  from  a  pedagL/c  i(:'al  viewpoint,   since  these  r'atios 
help  pinpornt  troub io  areas  within  the  students'  learning 

sequexice.     It  should  also  be  noted  that  a  hypothesis  of 

-  -  '        J,)  ■ 

task  dependency  can  not  be  inadequate  and  inconsistent 
simultaneously. 

The  apf)licability  of  the  Waibesser  procedures  are 
manifested' v/ithin  the  pedagogical  uses  of  the  hierarchy  as 
well  ;as  in  the  determination  vof  the  ^^alidity  of  the  learning 
sequence.     But  Vvliat  questions  does  one  ask  concerning  two 
valid  learning  hierarchies,    each  designed  to  achieve  the 
same  terminal  behavior? 

Magnitude  of  Kierarchy 

If  there  exist- two  equally .valid  learning  hierarchies 
each  designed  to  achieve  the  same  terminal  behavior,   an  I 
obvious  consideration  would  be  one  of  time  efficacy.  VThich 
hierarchy  can  be  traversed  in  the  least  amount  of  time? 
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Although  it  is  almost  cxioinatic  to  state  that  each  individual 
■traverses  a  hierarchy  with  different  tirpe  rates,   an  "average 
traversing   ti.r.e"   for  the  ;,.cpulation  with  which  the  hierarchy 
was  validated  could  be  associated  with  the  learning  sequence. 
But  the  aeneralizabiiity  of  such'  a  factor  is  low  without  a 
specific  description  of  the  population  v/ith  which 'the  hier- 
archy was  validated.     Hence,   xt  seems  reasonable  to  consider 
the  complexity  of  the  structure  itself. 

For  each'  hypothesis  of  task  dependency,  the  number 
of  subordinate  behaviors  may  be  considered  the  "complexity 
of  the. hypothesis";  with  reference  to  the  hierarchy  in 
Figure  4,  the  complexity  of  hypothesis  H(3;    is  two.     Let  N 
equal  the  number  of  hypotheses  in  the  learning  structure. 
Therefore,   for  any  learning  hierarchy,    a  complexity 

coefficient,   desigr.ated  cc,  can  be  coraputed. 

-  N 

y      complexity  of  hypothesis  (i) 

u  f 

complexitv  coefficient  =  cc  -   ^    .  „ 

N  ■ 

The  complexity  coefficient  denotes  an  average  number  of  / 
subordinate;  behaviors  for  each  hypothesis  in  the  hierarchy. 

Another  consideration  of  the  structure  of  a  learning 
sequence  is  the  nuiTiber  of  subordinate  levels— that  is,  the 
length  "1^'  of  th3  hierarchy.     The  length  of  the  hierarchy  in 
Figure  4  is  three  •     In  suinmary  with  reference  to'  Figure  4 :  , 


hence,    it  is  possible  b.o  associate  v/ith  each  learning 
hierarchy  a  vector  in  which  the  first  component  of  the  vector 
is  N,   the  second  coiuponent   is  cc.   and  the  third  component  is 
1.     A  learning  hierarchy  that  is  associated  with  the  vector 
^^1'    ^^1'  then  said  to  be  of   "higher    .agnitude"  than 

a  learning  hierarchy  associated  v.;.th  the  vector  (N  ,  cc  ,  1  ) 
if  two  of  the  following  are  strict  inequalities: 

^  ^V  '  ^^1      ""^2  '  ^1^  ^2  • 

By  associating  such. a  vector  with  each  learning  hier- 
ar        one  at  least  has  an  idea  of  the  relative  complexity  and 
magnitude  of  the  structure.     It  then  seems  reasonable  to 
assume  that  even  though  the  two  hierarchies  may  be  equally 
valid,   the  one  of  lesser  magnitude  is  the  more  desirable. 

Cell  Cruciality 

Since  valid  hierarchies  of  relatively  lew  magnitude 
seem  desirable,  one  can  address  himself  to  the  .question  of 
decreasing  the  magnitude  of  a  valid  learning  hierarchy. 
(Plowever,  note  that  it  is  more  desirable  to  have  a  valid  . 
structure  of  high  magnitude  rather:  than  one  that  is  invalid 
but;  efficacious  in  the  sense  of  being  of  low  -.agnitude. )  One 
can  decrease  the  magnitude  of  a  hierarchy  by  testing  the  ' 
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cruciality  of  the  hypothes i:^ed  subordinate  cells.  Let 
Figure  5  depict  a  valid  hypotl^esis  of  a  hypothetical 
learning  hierarchy. 

fai 

lJ±3       'Jl^J      IJI-J  l~i 

Figure  5  .  ■ 

?Si  Al leged  Val id  Hypothes is  of  a 
H  y  p  o th  e  t  j.  c  a  1  L  e  ar  n  i  n  g  H  i  er  ar  ch  y 

Are  each  of  the  subordinate  behaviors  in  the  hypothesis 
crucial  for  acquisition  of  the  terminal  behavior  in  the' 
hypotliesis?    The  tactic  used  in  testing  the  independence  .of 
a  set  of  postulates  -  in  an  axiomatic  system  is  applicable. 
For  example^   traverse  the  hypothesis  giving  special  atten- 
tion to  withhold  instruction  in  cell  d,   then  administer  the 
competency  measure.     If  the  hypothesis  is  equally  valid  • 
without  instruction  for  cell  d,   then  it  can  be  assumed  that'  v 
the  behavior  in  cell  d  is  not  crucial  for  acquisition  of  the 

terminal  behavior  of  the  hypothesis.     Hence,   cell  d  should 

..       .  .J         ,  ■ 

be  removed  from  the  structure.     In  so  doing  the  validity  of 
the  hypothesis  is.  not  changed  but  the  magnitude  of  the 
structure  itself  is  reduced*     The  procedure  is  then  iterated 
for  the  other  hypothesized  subordinate  behaviors .  .  ; 
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Conclusion  .'         .  • 

The  structures  of  learning  hierarchies  are  einbedded 
^   Within  graph  and  lattice  theory.     Birkoff    (1948)  has  stated  • 
that  any  organizational  d-nrt  is  essentially  a  partially    '  '" 
ordered-^ef;    and  che  nathematics  dealing  with  such  structures 
is  truly  aumzmg    (Lieber,    195^}.     However,    the  intent'  of  - 
this  article  was  to  consider  na.n..erical  procedures  which-  one  , 
can  effectively  use  to  detemine  the  validity  of  curricula; 
developed  as  a  hierarchical  learning  sequence . '  Hopefully, 
the  utility  of  the  results  from  lattice  theory  will  be 
incorporated  into  learning  hierarchy  -.search. 

Questions  related  to  the  effect  of  hierarchical 
learning  sequences  on  attitudes,   problem  solving,  achieve- 
ment,  retention,   ai^.d  transfer  are  being*  asked.  The 
numerical  considerations  discussed  in  this  article  will 
hopefully  help  researchers  in  this  area  of  curriculum 
construction. 
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SURVEY- SAI^IPIE 

Uaua-lly  a  few  observtitlon::  or  tests,  questions,  etc-  are  given 
"bo  carefully  selected  subJectG.     The  key  to  good  v/ork  here  .is 
efficiency,    Fre-testing  of  iruJtriurients.  cai-'eful  definition 
of  the  poT>u:Uvtion(vs)  5  and  careful  calculation  of  : r^ample  sizes,, 
confidence  liiriits  for  results^  t-^nd  consideration  of  various 
tj'pes-  and  con^>ji nations  of  caiu::^lirig  planfj - 
This  type  research  can  be  the  most  immediately  useful  in 
day-to-day  decision  waking  by  school  acuninistrators 
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Personalized  Educatw7i 

m  Soiitliside  Sdwol 


The  concejjt  of  miividjhilhfd  m- 
T  struction  covcvs  Ji  wkle'nmrvc  of  ef- 
forts to  tnilor  tlie  educarionai  pivjgr;ini 
of  the  schools  to  the  individu-j  lc-irner 
—to  his  achievement  and  his  rate  of 
development  (I).  Tlie  litcratarc  on 


child  and  adolescent  development 
shows  that  normal  growth  and  devel- 
opmcnf  occur  at  various  rates  from 
year  to  year  and  vary  considerably 
from  ch.iid  ro  child.  Educational  pro- 
grams based  on  statisticallv  derived 


patterns  of  average  growth  and  devel- 
opment are:  l)onnd  to  be  out  of  step 
with  the  devclopnient;iI  'jrid  growth 
needs  of  most  of  rlic  individimls  h\ 
any  given  group  of  children.  Longi- 
tudinal studies  have  shown  that  a 
great  many  different  patterns  of  de- 
velopment regular!/  occur  nnioni:^ 
children.  Some  of  these  diffcrciK^cs 
are  influenced  by  genetic  or  consti- 
tutional factor:,  while  others  iirc  gov- 
erned by  environmental  settings  and 
contingencies.  Children  from  difi'cr- 
ent  cnlmral  backgrounds  arc  social- 
i/ou  and  acculrur:ued  ill  various  wiiy/^: 
and  respond  in  different  ways  to  tlie 
.school  culture.  In  the  area  of  cogni- 
rion,  each  child  brings  to  the  class- 
roon^*  a  private  history  of  pnst  experi- 
ence that  shapes  his  current  patterns 
'^f  thinking  and  learning.  The  intrica- 
cies of  individualized  patterns  of 
grower  and  response  to  tlic  school  en- 
vironment make  it  impossible  for  any 
teacher  to  plan  and  prescribe  a  pro- 
L;!:;un  of  instruction,  that  would  be 
camj>Ictcly  appropriate  to  the  individ- 
ual cducarionril  needs  of  each  child  in 
her  class.  ■ 

Even  Though  it  n^.ay  never  be  pos- 
for  a  teacher  to  pinn  and  pre- 
scribe a  school  program  that  would 
allow  for  maximal  I  en  nune  by  each 
child  in  her  class,  many  teachers  have, 
found  it  fcosible  TO  novc  away  from 
a  :nii.fornr  insrruct'ibn/il  program,  to 
which  all  children  n^nsr  ndjust^  to  a 
curriculum  that  takes  some  accouut  of 
the  various  rates  of  dcvelopuient  and 
levels  of  achievement  in  their  cbsses. 


Personalized: CDucATtoN  lei 

These  ntrcmpts  by  teachers  to  meet 
the  individual  needs  t)f  cl;ildrcn  have 
taken  mnny  forms,  anti  new  adminls- 
tnuivc  insTruccionnl  arrnngcmcnrs  arc 
cc>nrinual!v  l^einf^  invejucd.  Amone 
the  most  \vcll  known  have  been  abil- 
ity-  grouping,  interest  grouping, 
achievement  grouping,  •  individual 
contracts,  tutoring,  emergent  curric- 
uhuns,  unit  tCiiching,  project  or  ac- 
tivity programs,  und  ])rogrammed  in- 
struction. 

Each  of  thi'se  arrangements  has 
been  tried  out  with. enthusiasm  atone 
rintc  or  another  as  a  possible  solution 
ro  ri^.c  perennial  problem  of  individ- 
uaii'/.cd  education.  However,  the  his- 
tory of  educational  innovation  has 
shown  that  no  one  arrangement  has 
yet  completely  succeeded  in  meeting 
all  the  individual  needs  of  the  diver- 
sity of  children  in  tlic  public  schools. 

Diagnosis  and  prescnpfion 

^Onc  basic  problem  underlies  all  the 
cfl'orts  of  teachers  and  administrators 
ro  modify  the  current  cducarional 
programs'  of  ■  tiie  public  schools  to 
meet  the  needs  of  individual  children. 
That  is  the  problem  of  ndctjuatc  diag- 
nosis of  needs  at  each  point  in  the  ed- 
ucational development  of  an  individu- 
al child.  A  closely  associated  problem 
is  rhc  problem  of  treatment.  Not  only 
rimst  diagnosis  be  accurate,  but  treat- 
menrs  appropriate  to  each  of  the 
learning  problems  presented  by  chil- 
dren must  be  workcxl  our  and  tested- 
so  that  a  teacher  can  know  which  pro- 
cedure to  use  once  she  has  recoo-nized 
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the  difficalry  a  child  niiiy  be  having,  of  ihc  curr' -iiium  presented  ir^  the 

Those  who  h;ive  watched  the  field  schools  is  inherent  in  most  programs 

tesring  of  Individuaily  vPrescribed  In-  of  individuah'/.cd  instruction;  The  re- 

stniction  developed  by  Robert  Glas-  hictance  of  scliool  aiuhorities  to  share 

ser,  Joseph  Lipson,  and  others,  of  the  control  over  the  educational  expcri- 

Pittsburgh  Research  and   Develop-  ences  that  are  provided  students  im- 

nicnt  Center,  arc  aware  of  the  trcmen-  dcrlies  nTucIi  of  the  current  unre^^v 

dous  compicxities  involved  in  dlagnos-  among  students  in  our  schools  and 

ing  individual  cognitive  performance  colleges.  As  long  as  school  authorities 

in  a  structured  area  such  as  mathcniac-  insist  on  mainraiiung  direct  conir.*; 

ics  or  language  and  in  prescribing  ap-  and  refuse  to  share  in  the  process  of 

propriate  educational  encounters  for  diagnosis  and  selection  of  appropriate 

individual  learners.  learning  experiences,  individual  srn- 

Even  if  accurate  diagnosis .  and  dents  Aviit  be  frustrated  by  the  v\^-y 
appropriate  assignment  of  learning  srruction  presented  them.  What  i. 
experiences  arc  possible  iti  highly  relevant  cognitiyely  ro  one  leanivii  i> 
structured  fields  such  as  mathematics,  irrelevant  to  r?iv)thcr;  and  no  teaclicr, 
language,  or  science,  there  still  remain-^  however  ^vcil  trained  and  however 
problems  in  working  out  school  man-  experienced,  will  ever  be  able  to  an - 
■  agement  systems  ;  that  AviiL  use  pro-  ricipatc  how  jmv  particular  program 
grammed .  materials  keyed  to  diag-  of  experiences  provided  by  the  school 
nostic  or  evaluative  measures.  With-  will  transform  the  thinking  of  each 
out  seeking  to  convey  the  idea  that  individual  lenrncr.  The  demand  b\" 
efforts  for  diagnosis  and  prescription  students  to  share  in  the  decisjon-mak- 
of  appropriate  learning  experiences  ing  process  in  the  educational  cnrcr 
should  in  any  way  be  curtailed,  I  feel  prise  stem*-  mainly  fronv  tha  frustnv- 
rhat  alternative  approaches  should  be  tion,  they  experience  as  they  are  re- 
pursued.  They  may  prove  to  be  u'Vorc  quired  to  pr.i  ticipatc  in  meaningless, 
fruitful  and  less  expensive.  trivial,  or  redundant  curriculum- of- 

Underlying  the  development,  of  ferings. 
most  continuous  progress  plans,  as  - 

Avcll  as  the  Individually  Prescribed  Parfkipafhn 

Instruction  syStem,  is  the  assumption       Any;  innovation  that  purports  to 

that  the  teacher  or  a  teacher's  aide  or  meet  tlu^  needs  of  individual  students 

some   other'  representative   of   the  in  American  education  today  must 

scliool will  remain  in  a  dependent  re-  take  into  account  the  increasing  dc- 

larionship  to  the  authorities  of  the  mand  by  students  that  their  educa- 

school  who  administer  the  diagnosric  rional  experiences  be  relevant.  To  a_s- 

and  prescriptive  procedures.  sure  relcvancev  all  tbachcrs  must  be 

The  concept  of  teacher  control  past  masTcrs  at  accurate  di^^^  of 

over  the  selccyon  and  the  sequencing  jmiLvitUml  levels  of  achieve 
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of  development/ cultural  Uackcrrounds 
and  personal  patterns  of  co<rn*rivT  dc- 
ycJopment.  Teachers  must  also  be 
aware  of  educational  alternatives  so 
that  rhe  choices  they  make  will  be 
enimently  appropriate  for  individual 
learners.  That  any  sucii  state  of  affairs 
will  ever  be  possible  in  American  ed- 
ucation is  quite  unlikely.  WIku,  then, 
can  school  administrators  and  teach- 
ers do  to  meet  the  deniand  for  educa- 
tional relevancy? 

Students  themselves  have  provided 
a  clue.  Their  argument  in  a  word  is 
"participation."  If  one  listens  to  stu- 
dents these  days,  their  arguiiient  boils 
down  to  the  assertion  that  only  the 
learner  himself  can  experience  the 
relevancy  of  his  own  environmenral 
encounters.  If  one  grants  that  the  stu- 
dents' assertion  is  valid,  wliat  is  the 
role  of  the  school?  How  much  should 
students  share  in  decision-making 
about  curriculum  experiences?  With- 
in what  limits  and  with  rcgii:^  to 
what  goals  should  decision-making  by 
students  be  permitted?  How  might 
school  organizatio'ns  be  resrructured 
to  permit  greater  freedom,  responsi- 
bility, and  decision-making  by  stu- 
dents?"^'";/ 

The  answers  to  these  questions  are 
not  readily  available,  and  only  after 
experimentation  with  new  systems  of 
educational  organization  and  instruc- 
tional programming  will  tentative  an- 
swers be  found.  The  evidence  is  clear 
that  answers  are  needed.  Students  de- 
mand-more freedom,  want  more,  re- 
sponsibility, atTrJ  insist  on  their  riirht 
to  participate  in  decision-making  that 
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afTircrs  their  cdriCAvional  opportuni- 
ties, occupncions,  c:UTcrs,  ami  sense  of 
well'bciner. 

An  experiment 

A  program  of  indiviclunli-/.cd  in- 
srrucrion  that  permits  continuous 
progress  wkh  increasing  degrees  of 
freedom,  responsibility,  and  decision- 
making is  underway  at  the  Southside 
School  in  Durham,  North  Carolina  as 
part  of  a  l]vc-)'ear  expcrimenral  pro- 
gram for  disadvanraged  children.  The 
progran)  at  the  Southside  School  in 
Durham  is  part  of  the  Durham  Edu- 
cation Improvement  Profjram,  a  Ford 
Foundation  funded  suicly  of  the  de- 
velopmental patterns  of  about  two 
hundred  children  growing  up  in  low- 
income  settings  (2) .  The  study  is  ex- 
amining the  effects  of  various  types 
of  educational  intcrvencion. 

.One  intervention  at  the  Sonthside 
School  is  C'^lkd  l\rso7fdi'Zcd  Ed^^ 
tmial  Vrograifruihig.  lt  is  designed  to 
personalize  the  educational  program 
to  the  extent  that  each  child  partici- 
pates in  decision-making  concerning 
his  daily  educational  program  with 
increasing  degrees  of  freedom  and  re- 
sponsibility. 

The  Perso!:ialized  Educational  Pro- 
gramming experiment  at  Southside 
School  is  based  on  educational  con- 
cepts developed  during  the:  1 920's  and  : 
the  1930*s  by  Carlecon  Washburne  in 
Winnctka,  Illinois  and  Helen  Park- 
hurst  in  Dalton,  Aiassachusetts.  The 
Dalton  School  in  New  York  City 
continues  to  operate  under  the  Dalton 
Plan.  In  both  the  Winnetka  and  the 
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Dalton  pinns,  students  were  cxpccred 
to  undertake  assignments  or  contracts, 
and  A^'orlc  at  them  at  their  own  rates 
for  as  long  as  necessary  to  complete 
them  within  an  ailoited'or  contracnial 
period.  Individual  learners  were  given 
increasihg  responsibility  as  they  dem- 
onstrated the  ability  to  complete  con- 
tracts on  rime  or  to  meet  deadlines 
for  specific  assignments. 

In  the  Dalton  Plan,  as  operated  in 
Ne\y  Cork  City  during  the  I950's, 
students  were  given  monthly  assign- 
ments that  were  uniform  for  a  specific 
grade  !evel.  Since  the  Dalton  School 
enrolled  children  from  high-income 
families,  uniform  assignments  based 
on  grade-level  standards  Av^ere  given. 
Where  individual  differences  in  "pupil 
ability  made  it  unlilcely  that  a  particu- 
lar child  would  be  successful  in  com- 
pleting nn  assignment,  teachers  were 
given  the  authority  to  lower  their  e.v- 
pcctations  a jkI  modify  die  assignments 
for  that  child  so  that  he  could  con- 
tinue to  participarc  with  the  class  as  a 
•whole.  However,  most  parents  found 
the  slowly  developing  pupils  needed 
ro  be  tutored  (out  of  scliool)  to  keep 
r'lem  from  falling  behind  the  school's 
achieycnienr  standards: 

In  a  series  of  adapta:ions  of  the  bas- 
ic Dalton  Plaji  tried  out  by  the  author 
in  a  number  of  publicjsdiools  at  the 
third-,  fourth-,  seventh-,  and  eighth- 
grade  levels,  it  was  found  that  the 
uniform  assignment  system  M'as  inap- 
propriate for  most  public  school  situ- 
ations. A  modified  Daion  Plan  was 
worked  out  at  the  Hopland  Elemen- 
tary School  in  Hopland,  California. 
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Under  the  plan,  students  participated 
in  choosing  activities  and  tasks  appro- 
priate to  individual  levels  of  achieve- 
ment and  areas  of  interest.  In  the 
Hopland  program  the  contractual 
idea,  taken  from  Carleton  Wash- 
burne's  work,  was  incorporated  into 
the  ba.sic  Dalton  Plan.  The  Hopl  and 
Plan  proved  feasible  in  an  ungraded 
seventh-  and  eighth-grade,  combina- 
tion class  of  thirty  children  in  which 
the  level  of  achievement  varied  from 
Grade  2  through  Grade  13.  The  pu- 
pils in  the  Hopland  class  were  from 
unemployed  families  on  an  Indian 
reservation,  froni  farm,  families,  and 
from  business  and  professional  fami- 
lies in  the  community.  The  Hopland 
program  was  designed  to  be  con- 
ducted  in  a  self-contained  classroom 
by  one  teacher,  but  it  provided  the. 
basic  rationale  and  the  structure  of 
Personalized  Education  Programming 
at  Southside  School,  which  enrolled 
about  si.xry  children  aged  sbc,  seven, 
and  ciMit. 

The  Soufhside  Plan 

The  si.xty  boys  and  girls  were 
grouped  into  four  family  groups  or 
' :})rides,"  which  mec-from  8: 30  a.m. 
'^'^  9:00  a.m.  for  planning  and ^  m 
again  from  time  to  time  during  the 
day  as  a  group  whenever  a  group  ac- 
tiviiy,  such  as  a  field  trip  or  a  physical '"' 
ediicarion  activity,  warranted  their  as- 
sociation as  a  total  group.  A  pride  was 
made  up  of  children  from  each  of  the 
flirce'  age  groups  represented  in  the  . 
primary  school  and  was  carefully 
composed  CO  fonn  a  heterogeneous  . 


■    ^^H^  ^')<^scn  Oil  rhc  lv.^^is  of  ncndcijuc 
achicyciiicnr, .  :ratc:  of  Ic.iriiiiiiV,  scs, 
rape,  and  degree  of j ;^ociij!i/;itiun/^: ^ 
.  :  the  first  year  of  t he  ]n lot ; pl-og r ;ii n  tw  o 
:   prides  were  con^poscd  bf  Vion-rendcrs, 
and  t\vo  prides  were  cotnpfjscd  of 
,  children  :\vho  Jiad  s(Mnc  skill  iii  rerid- 
■  ing. :  This  :Vcc()nHnotl;irio  ahilif.)' 
grouping  ,\A-as  inndc  :is  n  rrnnsitioiuil 
step  to  the  current  y car,  m  Iicii  cnch 
pride  has  representatives  from  i he  full 
range  of  abilities. 

■  '-I'fT^  Southside  Personalized  Educa- 
tional Program  jxgins  at  8:30  a.ni. 
with  each  of  the  ciiildren  going  CO  his 
,  pride.  Froin  8:  30  wncil  V: 00,  each 
child,  plans  his  d.iily  schedule  with 
:  the  assistance  of  his  pride  teacher  or 
:  a  teaclicr's  aicic.  To  coni}ilere  his  plan 
for  the  day,  the  child  cxaiiiiitcs  several 
posted  schedu ies  a nd  pays  a rcenrion  to 
sjiccilic  retiinremcnts  or  constraints. 
Iiach  day.  the  pride  rdichcr  lists  on  a 
conference  schedule  the  conferences  : 
that  each  child  is  c\pcctcd  to  attend 
durifig  the  day.  Son  ic  of  these  she  \vill  i 
;  conduct;  others  will  be  the  rcs})onsi-: 
bility  of  other  tcacliers  in  the  sclnjol, 
but  each  child  is  e.\|>cctcd  to  exaininc 
the   conference  schedule   and  put 
down  on  Jiis  daily  plan  opposite  the 
appropriate  time  the  location  and  rhe 
subject  of  the  conference.  Once  he 
Jias  e.Kamined  the  conference  schedule 
and  placed  tlic  conferences  on  his 
daily  plan,  he  is  ready  lo  begin  to  plan 
his  open  laboratory  rime.  Constraints 
on  his  frcedoni  are  li.stcd  on  the  wall, 
on  a  chare  rack,  or  at  die  bottoni  of 
his  daily  plan  sheer,  f  n  general,  these 
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;  constraiius  iinpose  on  him  the  .  rc- 
cuiirenicnc  riiar  1,0  >pi-iul ar  ic;K(:  a  half 
Ivou ;  cac 1 1 :  ( 1 1 j  1 1 la t he li in t  i cs,  rcUi ii Vg, 
w  rittng,. and  spelling  jtracticc  and  an 
:  additioiul  half  iiour  in  a  prt>jcct  ac-; 
;  ii\-iry  in  eirhcr  so^-iLi;^  siiu- 
:  sic,  (ir  science.  Simie  of  the  scheduled 
:  conlcrcnccs  w  iU  satisfy  rctjuireinenrS  ' 
in  one  or  anoihcr  of  ihesc  acadcihic 
areas.  If  so,  rise  child  ma\-  ninh  ::iiat 
a!ca.frt)in  hi.s  platuung,  Whc\]  he  has ; 
conipkrcd  his  thiiiy  pian.  ail  tlie.activ-. 
irics  (.»u  rhe  rcquji-cd  lisr  arc  to  be  in- 
eluded  suniCAUiere  in  ius  |.)ian.- 


Guidelines 

Lists  of  p()ssil)Ic  wa\  s  in  which  a 
child  may  satisfy  time  rei]  uirenicnts 
in  inaihemarics,  j'eading,  writinn',  or 
^P'^l/'^S  P';'^^rice;are  posted.  Suggested 
projects  in  social  stMdics,  arr,  nuisic, 
and  science  are  als<>  iisred,  but  a  child 
.  fiy.'^lil*^!-'}"^'  '^'^  ow  n  pj-ojects  from  so- 
cial ■'^fvitiics  or  science  units  that  teach- 
ers iniroduccd  ro  iiiiu  tluring  group 
co-nfereiiccs  in  social  studies  or  sci- 
t^ncc.  fn  facr,  teachers  encourage  pu- 
pils to  pick  individual  projects  that 
\vi!l  complement  rile  unit:  of  group 
study  hi  soc!;|l  snidies  or  science. 

Work  stations  and  traffic  control 

In  completing  a  daily  plan  each 
child  c.x'amiiK'S  the  "w brk  station 
schedule,"  This  scliedulc  lisis  the 
times  that  each  classroom  or  labora- 
tory is  open  and  die  number  uf  chil- 
tlrcn  pcrmittetl  to  sign  up  f or  each  of 
f|^«>;iations  at  any  given  hour:  In  de- 
signing h is  da i iy  schedu le,  the  pu pil 
picks  a  work  svaiion  for  each  period 


erIc 
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aig  the  (i;iy  unci  inserts  his  nnmo . 
opposite  the  work  station,  provideii 
th.1t  the  nn\innmv  numl)er  allowed  at 
the  STation  hns  nor  {)ecn  reached.  lie 
continues  to  pick  activities  and  work 
srarions,  plncing-  them  on  his  daily 
plan  shceriuuil  he  has  filled  in  all  the 
;  open  tin^e  sh)ts.  1  fc  will,  of  ciun'Se,  in- 
diuie  time  t\)r  gohig  ro  III ncli,  physi- 
cal education,  and  other  routine  ac- ; 
tivities  he  is  expected  to  take  parr  in.  \ 

When  ri  pupi!  Incompletes  his  daily 
plan,  he  presents  it  to  his  pride  readi- 
er or  n  reacher  aide  for; approval,  'rhc 
signature  of  cither  -one  indicates  rhat 
rhe  plan  has  been  approved.  After  the 
pin n  is  cleared,  the  pupil  begins  liis 
activities  for  the  day.  Throughout  the 
day  he  is  expected  to  have  his  dariy 
plan  sheet  with  him.  All  the  teaciiers 
in  rhe  lirograui  arc  encou raged  to  ex- 
anune  the  daily  plans  of  any  child 
who  come:;  to  a  work  station  and  to 
place  on  the  plans  a  syn)l)oI,  such  as  a 
st;\r  or  a  sor  of  initials:  to  indicate  tt) 
tlie  child  and  ro  the  pride  teacher  rhat 
the  pupil  has  arri\-ed  ar  a  particular 
station  on  scl^edute.  Sinjilar  symbols 
are  placed  on  the  daily  plans  to  indi- 
cate display  of  appropriare  study  hal)- 
its,  work  underway  at  the  f)roper 
tiivic,  or  compfercd  on  schedule.  Posi- 
tive coinni^Mits  for  quality  work,  cre- 
ative ideas/or  products;  or  developing 
skills  are  also  entered  on  the  plan 
sheets,  : 

Allowonces  for  differences 

Since  decision-making  :  and  phm- 
ning  arc  complex  pr(^cesscs  that  nuisr 
be  learned,  children  arc  given  small 


degrees  ()f  frecdoni  at  the  bcginnincj  : 
of  the  progfanVin  the  falk  The  first 
two  <M'  three  weeks  teachers  or  aides , 
plan  the  children's  progrnms;  much  as 
in  a  conventional;  teacher-directed 
program.  After  pupils  become  farhil-  ] 
iar  with  the  work  stations  and  with  ■ 
the  '■dail)'^  plan/'  children  who  sho\v 
the  abillry  to  read  time  and  to  irovcrn 
themselves  in  a  responsible  nianner 
are  given  a  half-hour  for  an-  appro- 
priate activity  of  their  choice  and  n ; 
jMacc  to  complete  the  activity.  A:fter 
a  child  works  effectively  for  a  week 
or  so  with  one  half-hour  slot  available 
for;  clioice-making,  additional  half- 
hour  periods  arc  made  available  grad- 
ually—consistent  with  each  cJiild's 
abihty  to  govern  himself  and  to  oper- 
ate with  greater  freedom .  In  all  cases, 
hoNvevcr,  the  child's  choices  arc  con- 
strained by  the  guidelines  posted  in 
his  pride  and  listed  on  his  plan  sheet. 
In  addition,  each  pride  teacher  holds 
routint:  wcekl)'  or  biweekly  confer- 
ences \\Mth  individual  pupils  to  review 
thei  1^  da  ily  ph  ns  (for  the  past  week  or 
tAvo)  and  ro  make  recqrnmcndations 
on  h(Av  their  plans  and  tiieir  perform- 
ance niight  be  improved,  ^  In  :  some 
cases  some  freedom  (a  few  time  slots) 
may  be  taken  away  for  a  week  or  two 
until  pcrforinance  is  improved.  In 
other  cases  increased  freedom  may  be 
awarded.  The  pride  teachers  keep 
records  of  individual  pupil  confer- 
ences, and  the  records  form  the  bnsis 
of  parcnr  conferences  as  well  r  >  jiidg- 
ments  regarding  specific  insrractional 
j)rograms  needed  for  individuals  and 
small  groups  of  children  in  the  prides. 


Reporting  to  parents  and  to  pupils 
is  based  on  the.  daily  plan  sheets  and 
the  progress  records  kept  by  the 
reachefs*  The  progress  records  accii- 
niulatcd  by  the  teacher  through 
weekly  or  biweekly  conferences  arc 
supplemented  V  by;  standardized  tests 
and  other  evaluative  h.^truments  as- 
sociaccd  with  prograniincd  niatci  ials. 

When  indiyid^ial  pupils  have  pro- 
gressed to  the  point  where  they  can 
plan  for  more  than  one  day  at  a  time, 
a  weekly  plan  sheet  is  used  to  Jet  pu- 
pils work  out  projects  that  will  take 
several  days  to  coniplete.  Most  pupils 
in  the  middle  grades  will  eventually 
use  weekly  plans  or  even  monthly 
plans.  When  pupils  arc  usnig  weekly 
or  monthly  plans,  it  is  possible  to  in- 
corporate out-of'School  activities  into 
the  over-all  instructional  program  as 
individual  pupils  see  the  relevance  of 
out-of-schoolV  activities  to  their  in- 
school  projects. 

■■  Aid&      v.'-'  ■      ■ .. 

Personalized  Educational  Program- 
ming at  Southside  School  is  extremely 
flexible  and  permits  the  involvement 
of  supplementary  equipment,  ■  mate- 
rials, resource  centersv  and  specialized 
personnel.  To  enrich  the  curriculum 
offerings,  children  may,  during  labo- 
ratory time,  use  tape  recorders,  head- 
sets, phonograph  records,  games,  a 
variety  of  projectors— filmstrip,  car- 
tridge, slide/  8  mm.,  and  overhead— 
and  other  materials  and  automated 
ec]uipment.  Special  conferences  are 
scheduled  to  intrpduce  the  mechanics 
of  operating  new  equipment;  then  in- 
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dividual  children  '.ire' instructed  and 
tested  in  the  use  (^f  the  equipment.  A 
list  is  made  of  pupils  who  are  free  to 
choose  to  operate  the  equipment  on 
rhcir  own  during  laboratory  time. 
()rhcr,  less  skilled  children  are  assisted 
by  an  aide. 

A  variety  of  prc^gramnicd  materials 
such  as  reading  laboratories  and  Indi- 
vidually l^rcscribed  Instruction  math-^ 
ematics  arc made  av^ailablc  ro  the  chil- 
dren in  rlic  resource  center.  Special 
teachers  who  have  one  or  two  hours 
a  week  available  are  included  in  the 
program.  Their  available  time  slots 
arc  listed  in  the  work  starion  schedule/ 
Teachers  assign  sonic  of  the  children 
to  the  special  teachers;  other  children 
who  are  more  reliable  schedule  them-* 
selves  during  lab  time:  When  new 
teachers  or  new  equipment  arc  intro- 
duced into  the  school,  pride  teachers 
set  up  special  instructional  confer- 
ences to  discuss  the  new  laboratory 
opportunities  and  to  denionsrxate  the 
new  equipment. 

Balance 

X  Since. social  skills,  concepts,:  and  in- 
terpersonal relations  are  best  learned 
through  group  activity,  it  is  important 
that  pride  teachers  plan  the  social 
studies  program  carefully  to  involve 
all  the  children  in  the  school.  Many 
individualizeJ  programs  tend  to  iso* 
late  children  from  one  anodier  and 
reduce  the  amount  of  peer  interac- 
tion and  group  learning.  In  Personal- 
ized Educational  Programming,  pride 
teachers  plan  units  of  study  in  the  so- 
cial studies,  which  become  a  major 
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source  of  group  goals  jnd  acriviries 
Out  of  rhc  unics  of  stuciy  conic  iiinny 
smaM  groiif)  :uu\  indivKiiial  projccrs- 
that  ;iro  curried  on  in  laborritoiy  liiiic, 
buc  the  pvcr-all  goals  of  ; the  units  are 

:  sec  up  1  )_y  r!ie  t enchcrs.  Many  acii dcm- 
ic  skills  are  pmcriced  in  sociLiI  stiulies 

.projects,  but  ucsUc  'ic  skills  are  also 
raught  in  special  instructional  confer- 
ences on  .each  of  the  skill  areas. 

Prospects 

Th  e :  South  side  Sch  ool  ])i  lot  p  ro- 
granv\vas  in  effect  during  the  196S- 
69  school  year,  with  about  si.xty  chil- 
dren from  low-income  families  in 
Durham,  >JorthXarolina:  Abou  half 
of  these  children  continued  in  the 
program  during  1969-70.  / 

After  one  y e;u-  of  experienci;  in  the 
pilot  program  most  children  were 
al)le  to  handle  a  snljstantial  prirt  of 
their  own  educational;  activities  re- 
sponsibly.vA^any  of  the  children'  who 
are  iivour  disad\'ant:iged  group  were 
unreliable  at  first  and  needed  conscant 
"supervisioii  and  direct  instruction,  but 
all  bur  '>ne  or  two  have  made  .sio-nifi-- 
cant  progress.  The  greatest  problems  : 
occurred  where  noise  aad  movcmenc 
in  onc  work  area  interfered;  with  in-' 
strUction  in  ..adjaccut  work  stations. 
I'he  physicaMayout  of  the'  school  in 
] 969-70; separates  verbal  instructional 
stations  from  construction,  ■  art,  and 
dramatic  play  areas.  Distracting  move- 
ment and  noise  A\'crc  major  deterrents 
to  effective  communication  hi  the  in- 
strucfionai  centers.  ^  Vith  tl  ic  cur  rent 
P^^'^^*^^^  arrangement  of  instj  uctional 
areas  and  work  stations,  many  of  the 


ry  1970  ' 

problems  of  interference  can  be  over- 
come. ,  •  '  ;,; 

The  future  of  individualized  in-; 
struction  in  the  public  schools  dc- 
P'^"^;-^  "P^-'":  fl^c  ability  of  school  au- 
thorities to  develop  systems  which 
will  transfer  greater  and  greater  deci- 
sion-making power  to  children  within 
carefully  structured  limits  that  guide 
pupils  and  help  them  learn  ways  to 
govern  themselves  and /  achieve  on 
their  own.  The  problem  of  socializa- 
tion in  a  technological  society  has  be- 
come increasingly  complex  as  mobil- 
ity has  increased  and  patterns  of  em- 
ployment  have   been  transformed.; 
New  ediicational  systems  are:t]rgent- 
ly  needed ;  do  prepare  all  children  to 
govern  .themselves  and  assume  greater 
rcsponsii>;iity  in  planriiiig;  their  own  ; 
educati  t)ha]  experiences.  Any  system 
of  education  that  is  designed  to  be 
controlled  completely  by  school  au- 
rh ori ties,  however  individualized  it 
may  be,  seems  to  this  observer  to  be 
incompatible  \vith  the  economic  and 
social  imperatives  of  modern,  techno- 
logical America.  ;; 

Students  are  demanding  rclevancv 
in  ail  their  etlucational  ext)eriences, 
and  none  of  our  educational  institu- 
tions is  exempt  from  increasing  pres-  : 
sure  to  abandon  instructiohal  and  ad- 
niinistrati\-e  patterns  that  are  no  longer; 
appropriate  to  the  educational  needs 
of  ■students.; ;:;  ■'.:,;;,;;: 

The  innovative  program  at  the 
Southsidc  School  in  Durham,  North 
Carolina  is  one  of  the  many  possible^ 
arrangements  that  offer  increased  de- 
grees of  freedom  and  decision-making  ' 
within  gradually  broadened  Hmits. 


Learning  to  use  frecdonv  responsibly 
is  fundarhcntal  to  the  effective  acqui- 
sition and  use  of  knowledge  in  a  de- 
mocracy. 

RE  F  EREN  CES| 

I.  This  article  is  based  on  remarks  pre- 
:  scnted  at  the  s^nnual  convention  of  the 
National  Society  for  Programmed  In- 
struction Sheraton  Park  Hotel,  Wash- 
ington,       April  10,  1969. 
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2.  The  Durham  Education  Improvcnicnc 
Pi-(>gr3m  is  a  project  of  the  Ford  Foun- 
thition  under  rlie  aiispiccs  of  the  South- 
ern Associaiibnof  Collcgos  nnd  Schools, 

.  whose  Education  Improvement  Project 
is  funded  by  the  Ford  and  Danforth 
Foundations. The  Durham  Education 
Jmprovemenr  Prognmvis  jointly  admin- 
iscert-d  by  Duke  University,  North  Caro- 
lina College/Durham  City  Scliools,  Dur- 
hnm  County  Schools;  and  Operation 
Srcakthrough,  Inc. 
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ANALYTICAL  DIAGNOSIS  IN  ARITHMGTIC 
There's  No  Easy  IVay  to  Diagnose  Instruction 
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■KSSv^io^  'S'^S'-t-     P'-ia&es  or  eiein^i^ts  of  some.:  t;r;oce3R::iv]  urhj-h  ^  " 
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■K-ribxvledge  of  "basic  addition  raf 5 t'^V 
:i5!.S^''?®^^  ^-^  adding  ^vtiole  numbe-ci:. 


■ DIAQMOSTXC  TEST  in 
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Z9  +.6 

IS 
33 

8 

?  -  :  :: 

13 

29  -J- 

37 

8  - 

5  - 

6  - 

Addifeion  of  longer  coiumns 5 


4 
2 

1 


5 


83 


do 
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5 
9 
z 

1 


8 

9 
6 


59^ 

II 

52. 


Iii  avjalj-iilflg  the  r-e of  ■•eats- o:^  ,:'t-30ific  b1v;  }.1^  as  twO^s 

i.lst8d  p  spe-::iiaX '  v.,*iiift:i.derav;^      s^iouXd  be  given  fo  che  papil'  x 

i'-ate/of :  v^'oi'kiv  the  accur£..ey  o/;  iiis  r-r:opr-n;';e.?:7 tl-jt;  .level  of  difficulty 
at  vihioh  a  skill  'tevicis  t-o  h^-^sM  doviiip  e.rj<:l  e v:\rj.e ii'.-.e  of  43Lriy  t--9-c  ious 
difficulty  Nhlch  Qieavvly  r-^.:qu.l>.-a,2  :rur'r;ne     sxrves'Sj iiatioiu,     The  re.-;- 
iatibuHhlp  be  tween  rate -arid 'aeciivrv-^-c/.y  :l5:  a  i'  .i-vjri  in  ■•'-'as.  d:".,c-g;t-ajii  be'iov/-  ,. 


ACCURACY 


V. 


"5 


:  High  : 

High  Itt  bv  th-  5.'at5 
and  ac-::tu-.-3cy 

.        -  .  "  , 

Ays s^age  i.B  rate  " 
high  in  accuraciy 

high  iB.  .acourany; 


High  r-i't^i 


Low 
III 

■    VT    ■  ■ 

A-y e  j:  age  1  ix  i:a t- 
1 1;  -.■/  1    a.: ?.ou .  S-O y - 

IX-  , 

.'liOw  lii  boi".h  r-a--:-^ 


Thes.  ol-V-^ractarlstiGs  of  puplla  who  .  fall  into  eFAch  gi-oup  ean.:yn^?ad;0.y..:- 
be  desci'-iosd.     PupS.Is  In  Gro^^^^^  a.:;2  l;he  uios-:'.  abJ.e  crjes-.  ; 

Pupils;  In.  Group  XX  clearly  are;  the  leawt  ab3:5  ones;  i-i:-  wou].d  ui:idoubt'; 
ediy  be'  e:?;treme'iy ,  f  f  xcnalt  bo  .br.:.i  :ig  abou';;  any  cor^sidf^rable  impr-'0'?'C-- 
meat  In  :fchei-r  workn     Pupils  in  Qi,'  ;>up  III  ai^^  spaea.y  but:,  M'i^-^r.auv-ate, 

beoause^  of  :a  iac^k  of  b£iKla  ^kriowledge  av^a  .:;kil'is  requiri  ng  D^rther- 
diagnostic '  study  <>■    Pi;ipili;i  iii  ui^'Oap'  viT  are  e^::ricieriC5  v'-ai"e 
wo-?«keris5  iNThose  rats  of  vvrov.-k       :;il6^'^;  it  l;~  ■  qu,::-.t;;^;  pi-^ooable  that  tht-;tr  •, 
i^-ata  :of  ivTOrk  can;  ©e  inc5-c:a:£-£d  i:>y  ,we::j.-dir?:Cted  pra-it:lCi?P  bu t  : ac      - .  .., 
^•aay  :Shculd  r:ot  be  oaerlfif?ed  to  inches ra^^e  or  w-oti: V    ri^pi.ls  ji'j  ^  ^ 
Groups  li  s  ■  IV.,  V  j.:  and  VXI  pi-.>babl y  ,do  iitO  t  r>i:q,aire  av^y  bpeelaX  amt^o- 
ana©  othev*M:.han  giJidai^OiS  of  .regular  Xa-sty^ic^tion,  :  where  a -i  the  wo  x-^k  .  o  f 
.pupils  .ill  groups  VI,,  ■  VIXT,-;a'ca  tX  shouXd  he  fvvlJ,     a? -aX suited,  to 


.It  .is .  kmwrii  Usav  :ra't8  o:f  v,;o3r-k  :Ls  ar,  ivapi:--- ■■::ant  rsiu  bo;.-  c,ondi?;:.vo:ni:ttg  . 
^aoour^acy  of  rtispocu       on  tlie  ba^ic  nur.-u~e:t'.  f'ficrc,-? . :    The  correcv-^  re--, 
sponees  'for'  three  pupils  ofV  a  :j6S'^;  cC:  E5  dTtrs'i&uX't  nr<u:it.i  plicae  ion  . ; 
^  facts  adsiih,lsi;erad"  In  .fiY©  dif -r-'ii-ent/wayi??'  \-hat  aire"  gi-^^ew ■■'below show 

"tnis3c. ."■■■•.  .  '.  ,      ■  ■      :  ■  ,■■■,..■■ 


er|c 


Kethod  of  •Xes  ting  Pupil  I        PupxX  11      Pupl  :•  17.1 


Ay   Unii  raited  time  to  iMv-d  &n-iwei^-3..  .,.,„.     „     25  25-  ...  1.8 

Bo    F&Q.ts  dic/Ga'i;ed  .at  itite-evals.  of:  .  . 

.  -.  .( 1}  ;  5  seacnds,-  ,  >  ^  ,  r.     ?5  ''.-  ^  5,5 

iZ)    M'  seciOx^•:i^:,     .  ,  ,   .  ..  ,...«  ■>    :i:5  ._8  .  ■ 

■ '.'-{y)    2k  s<^(^onflB-, ......  ^  ,  ,\ 25.  ■   1 .2  .  .  i- 

C«    : Feats,  .on  te^t  paper  i;'£a.d  at  3■^Becor^6.  Ui-'cervB^iE  . 

•  ■  ■  .  internals .  ^ .     2^1  2 -J-  ■  ■         '  '  J^r 


: The.., answers  of  iPupii  I  all  correct;  on  -ch^r;  rive  tesjs:  r^esults-  ■ 

.i^re  :aot  af-fected  at  all  ,  by  dAf  rei-encss'  m  ahe  '  ra    s ■.       whlrjh  the 
.products  seem  w?-/..tten=     The  rates,  gl  ^en  v-iidei-  ■  B  -.'/s.-e  ■  c-^-.n trolled,  by 
..UBi%  ^?eca:=ijdlngs  of  .fae-^b  and  haNi'ing  ths  p;3pi'i:  vjvj. iva  r.'ne  pr'Odu:,-.'oG 
:.:dn  :.Equamd-;papei."^  as^  papldiy  as  -  the  .  facts'  w& re  d'vo  tated-.     'i*[>e  i  Gsi.i'.'vts 
;for  f^ip^^  work  out  all  v'ne 

.answQi*8;ijwhsaVh^         unlimited  txaej  howa-^-©rp  under  time  pre.isur^e  his 
■:re^p<>uee3.  'beca^  :  ae^curate ^  Indic-atir.g  iacik  :  of  raas  tecy  ai-id  a 

;.tei^eijc^\.to.  :gusss.:at:  a^^^  The  results  foi^  Pupil  ,111  .iiidiciate 

;..'^,.tmcoum;t^^ .. teorlt  .  even  I'i&en:  u-Gder-.:  110  time  pre s 3u;r-e  ^ ;  xn<i-r^!ssei5.  In'-- 
a.oo-ttra<'!y  ^/UTidsv:  mciderate  t  ime  p-i.'©ssui-e and  an  alnsost-  cyaip.le  te  .  'break-  . . 
...  dowii.,- when  'tlie  rarta  ;  of .  vs-l tirig  ,  ansviera  was  ii'ie-rGased  to  irrtev-^-alis  o.t 

^ Ideally f  :al2.  pugfils-  £lao..ttld  st.:c-i'«e  i-cltially,  'for  aocu rscy  and  tneri 
; :  gi-adual     ;  iciis'ea^e ^  th«  1  r  .ra te  3  o  f  viotk  w  i  t h.o va  t  sac  r l  f  1  c  i  >ig:  acou  x-acy ...  . 
,  The.,  rate:'.;. at;  f<hlch  difre^snt.  i-odi vid"aaX« ;  ?/.?:'.ll:,  work .  v^j  1 11  dsp-end^  on:  ■ 
©aoh^  oiae^':.3:.P!&r5s^^^^^    tralis  .and  ^CiViarastey'lstlcs  and  on  his  if?lllii2g- 
ae£fs"'.to;  ;raak:e  tlTfi;  effo'rts  iie-i^ded  to  aobie-re-  a  b:.a:h  Xe-vel  of  pej-for--  ' 
mana©-:.through  ;:listelllgeiti:tl  e£.£ec^i-<ye  ■  practice .  ^Iz  must  "bs  remem-- 
.■bei'efi:'..that.  :>.-ate:  vof  wot^k:  will  ,inci:^ear>e'"  with  aar;ur\t  lvy  si-M:  ther©.foi:-e 
.spsed  .  should  uot  ,  be  -unduly  .■effiphsai-aed:  esi'ly  ih  the"  laav-niTig :  process o 
.  Prof  IQ l«ncy  will.-'  iiic'/j.;'sa;se  Wrtdn^  pu i-po se f  u  1  ^  ■  sy s  terna tx e  pract lea  :0  f 
imeeningrul:  skiXl.-  la  paja-^ldad..    The^tei^chec^s  probl^Mn  is  :  tc  help  \  . 
•children  :i=riG  re  a  as-  the  -eO^afjiency  :  of  th«lr  thought,  habits  ..aud  jTiethcds 
of  work  B  .wh .ich  ,  wS^X  .auto dsatlc-al I y  Lead  .  to  ssc  e  j^ap i d;  i.  e  spotises   ,   . "  . 

iB.'-aoras  of  ..the  fficsfe  daraple-^:  ^phases  03'f  -ar-lfchnietict,.  ^ut^fi  as  diTisio;A 
by ■  two-.place. ;  r-ume^^^^        1 1  Iej,  posiislbls  to  adininis tev  a  ji-torxes  ot 
diag-nd s tic  ^ sterol the  -epeelfio  eleriient-c  1^^  the  to- 
tal; pro  cee  a  that  ma^^hs  ^  ooxit^lltxxtyov-y  to  an  appa>n-.v.!.t  de,;MGia>aoy  In  it. 
The-ie  diagaosfcie.  exercises  cau  be  eonLstruoted  so  a^j  L;£;  test  each  of  .. 
the  .basic  skills,  included  ;ro  the  analysUi  of  the.  division,  example  . 
gi'^en  on.  pa^e' 2:0|l  aa  folio va^  • 

; :  1 «..  _  AMllty  =^13 divide?  by  crie-place  .TiUSBher'S:,  as  an  lodicatlon 
■.  ■     or  ltia#«!deage^^  the:  ■  ^ 

.  .  2«- Aliplty:  to  -  eatiroa'fce..'  and.  place  -quotient  f  igiiregi  correatly j  ■  ■.  ' 
f  Irst  -^^se;  lia  thi^;  ■  e^itlmated  .  quo  tient-:  is  the  true  ^  quo  titint  o. 

■  ERJp^  in  :^;l^§3,:  ai^:  then^^y^se^-in  which --the  estimated  quo  tient '  nwst  . 


be  GOT  re  c  feed  so_as  to  find  ths  tras  q.uotle\Ttj  as  ir]  3^i'/23i  i 
3.     Ability  to^^:i^^^^  as  in  finding  products  of  divisor 


lie  fdllowiiig:  dia^ostlo  exerci        i IXa a ti^ate  the  approach^  for 
dentifyiwg  weakneBse  B  iti  di\ris  ion  by  two -place  nutcbers.^ 
1e^r<^' -q:^  <n\  any  ;  of  the  four  par 

f  'the'  testa  ia  dlf  f  Iculty '  that  should  be  care  fully  studied  may  toe 
;USpQO ted »;  ..■■';■■,::  •■■ 


:   V  DIAGNOSTIC  TESTS  TO  LOCATE  WE M  SPOTS  IK  DIVXSIOH  . 

Division  by  one -place  iiuTKbsrs  .- 

'  ,  '  ■     b,  ,  •  c  .        .  . 

X«  TV^^^^^^^^r     3^  6/^909  9/^238  8/^839 

Estimating  and  pXaoirig  quo  tie  at  figures; : 
Find  the  first  quo tl@nt  figure  onXyo 
B;©  8Ujre  to  plao©  it  Gorrectlyo 


a 


b 


a 


io  20/1376  ii,3/r£F98  83/502? 

Multiplying       >ieed©d  m  .di'«asions 


1.  3 


6 


25/792 


Not0o  .•  Check-,  products  r  check  then  on  subtraatioxi. 


)o    Subtraction  as  needed  in  divlsiori: 

Firid  th©  retftaiioders  pnly  in  these-  ex^iffiples 


to 


301 


6?^ 


'668 


.■  23^^ 


96/5803 
1 7/^58? 


96/766 


53/^7 


iioalytit?©.!  diagnosis  should  also  closely  i)amllel  the  de-felopraent 
if  aew  ^rko    At  regular  iu-tervals »  pref  erably  at  the         of  each 
aew  uttit  of  instiructloxip  a  eui  tably  aonstr-uated  deyelcpsaentBl 
il©5UoStto  test  should  be  admiriJ.at©i-ed  to  ^  ^^.asB  to  locate  weak 

spigj^^lia  the  new  work  that  has  bean  pi-asented  so  that  they  oau  be 


5 


:pi^mptiy  diagnosed  and  uorr©c\:.s do     Pc-r  ©xsiapAO ?  a  complete  aeries.- 
of  de-/elopineiital  diagiioatio;  tests  i'^a  addition  of  ^'i-acitions  shouxa 
parallel  the  folio v?:vng.  serJ.^^ 

r-  -  Uiii-t-  1»     Mdltaoti  Of  like  fv«.cv.ions,:  no  oaiTyJ-ng  required.,  as 

Unit-2v    Addition  of -like  fvachiOiiia.i  r^^^  of  in?p^=cpev 

fractions  and  carry:;,  ug  -»2  qui  red.  In  ::'lndt-ng  Kums;;  as 
•  in  k     ic  :  3/^^"  "*-  :3Ap  avjn,  ^  5/©      3  5/6o  ■. 

.    .    Unit  3<-.     Addition  6.i''  uul'lke'-. n-aistibns  with-  .demminafcor'S  in  the 
j      ,  &&)Sie  f  amily 

-  ■  .    tr-dit        ■  Addi-ti03fi  of-  uYiiike  iVaotions .  wi  fch  dehorainatops  i:io.t  ,.  in 

th©  same  .family,  as  In  l/  ^-^^  A.  ^-l-  3.^,^:      2      5 .  aad 
-   5  3/' 


/tj,  4-  6  5 /&  => 


Aaaiytioal .  aiae:nb8tio  teats  ar-e  goa^tiiaes. ...r<>v:ad^lS:tg 

.    1  c:.  ^^SeleGt  an  in  ejcansple.  .  Unlt- 

2:  in  addS.tiori  of  like:  rractioTjs  Brf  def  1  ^  . 

.   ,  =  Z„    Break  down  tJ.'si?? .  major  -.unl t  it-to  a  :: s@ ries  o£  the;  sub^^ypss  . 
taught  in  the'  course  o.ir  the  .ualt«  •   For  exaiTiplB.  the  serxes  oi-^ez-- 
f>.mplea  given  dqIck  ;  iX3.u.stra:.t'3S  the  3  vr^r^BM-n^  eoraple^f.!  ty  Giv  tne  . , 
basic  subtypes.,  all  of  them  taught --Sri.  th©  ■  •souy^.se  or  thiis  uiixx;. 


-7- 


1 


..  3.  '  the  d.lag-aosti<^  exerci^je  iiBelf  there  should,  be  .s^  row  of  .  .- 
at  least^  three*  Preferably  four ©xampl«^-£J  repre se-ni^s-Ug  eao.a  ox  -tne  .. 

exarap:.le  '  irt  a  siTsgle^^^E^^^^^^^  exarapXes  -may..  be  i.-«sga>;"d©d  as^an  _  . 

'ingle  rcr^tl^  a  pei'^3l!?:tent  diiTlc^^  .        .    \  ;  ' 


ifro    To  faoiix'tat©  rsmedlal  treatment,  de'^eXopwen.tal - .diaghoa-fc^o 
tests'  should  be  keyed^  ;to  special  reffiediaX /helps  and  learning  aids 
Inoluded  in  teiitbooksV  workbooks,.  02-  suitable  supplementary 
•inaterials  .prepared  by;  the  ;  teacher  r.: 
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.  Similar  ti^ste  can  .be  ptepa\-ed  icr  a .13.  otif.:^ar,v.:yii6  v;i -ah  who<e  ;.uffi.«  ■ 

bes-s,,  f^-actiotis,  and  d6caiiiaX;!o     The  :hvJc:  xrobXefn  Is  ho  01:^6 a k  dc-a. 

ai;  units,  .aiad  thea  to  i(ie^;•:^if  y:  oh-.  £';bryv.e:i  c.:  exs-rn^lec  o;jt:;n?it;o  ' 
the  procedure : 


\  Ujai  'c  ,  5.  s  dividing  by 

20/w..     20/63"      i.'o/7^"  . '  ■  ^^^^'O/riTv 

-  U-ait  2  ,,  dj.vid.i.-ug  by  oij.bo-/        .-■-p)  ;2v..-:. 


111  V  S  .:  v;!      iiCi   ■'.■  ;') 


2lftT  21/ yO 

•     Uultj  5 8  oo:i'T--ection  of  the  eaMmatsd.  quoti'evit  neaessav-y.- 


21/80""  2^92 


i9/23< 


3.;-/i35 


c 


DIAGNOSTIC  THST  IN,  ADOI'VION  0?   LIKE  i: FRACTIONS         TYPE  2 


■  CL 


3/ 


'9: 


4c 


He 


7." 


1 


^7 
'  to 


4?' 


a 


:  :  ^ 


3/.. 


7 


4/ 


H  4o 
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NAME 


SCHOOL 


DisgiiDStic  Test  l-;;  ^lu;  ■::v.pV;L:,  o.'  '^■.■u\b:r^  s 


213 


x*6 


x6 


29 


■b 

30' 

.  89. 

Sv5 


906 


6G  V  0 


Ccj.-,- 


■  >-  1 


:'2  a  ■■■-Two-  and  -  Th5;«ei5  .Flao'S  l^ul. t2^K-i^|^ 


-..-'a.  ■ 

■-  c 

a: 

U  ■  .18  ' 

■■■■■  30.- 

26 

Hi?- 

806  \ 

'  900 

5-  39? 

X2  59: 

3... 
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PRE-TEST  AND  POST-TEST  FOR 
PLANNING  FOR  EDUCATIONAL  DECISION-MAKING  IN  THE  70 »S 


I •  FundameBtals 


The  most  marked  difference  between  ^'basic''  and  ^'applied'*  research 
in  education  is  in  the  extent  to  which 

a-      control  and  precision  are  emphasized, 
b,      the  Hawthorne  effect  operates. 

implications  are  draim  concemi:  g  school  practice, 
d*      theoretical  constructs  are  tested. 

The  major  difference  between  pure  and  applied  research  is  the 

a-      use  of  research  design* 

b.  met!iods  of  population  selection, 

c.  stability  of  conclusions  reached. 

d.  application  of  the  findings. 

IVhich  one  of  the  following  defects  is  almost  impossible  to  remedy 
in  the  process  of  problem-solving  through  personal  experience? 

a.  Subjects  and  situations  are  probably  not  typical. 

b.  Assumptions  are  not  identified. 

c.  Meanings  are  not  defined. 

d.  Processes  of  solving  are  not  replicable. 

Wliich  one  of  the  following  ''methods  of  knowing"  is  most  characterized 
by  self-'Ccrrection? 

a.  tenacity 

b.  authority  . 
c-      a  priori 

d.  scientific 

Which  of  the  following  is  the  least  important  aim  of  "pure'^  research? 

a.  control 

;b.  prediction 
G •  systematization 
d.  application 

The  aim  of  theory  is  the  explanation  and  prediction  of  phenomena. 


a. 
b- 


True 
False 
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7.    ^-rtiich  one  of  the  following  best  describes  induction? 

a.  Specific  to. general. 

b.  General  to  specific, 
c*  Specific  to  specific, 
d.  General  to  general. 

Read  the  quotations  in  items  8-10  below*  l^hat  source  of  knowledge  is 
represented  by  each  statement?    Use  the' following  key: 

a.  authority 

b.  inductive  reasoning 
c«  deductive  reasoning 
d.  tradition 


8*    Mrs.  Adams  said  "I  know  that  the  children  in  the  Head  Start  program 
have  been  given  considerable  practice  in  reading  readiness  skills. 
Since  it  is  known  that  practice  improves  these  skills,  it  figures 
that  those  children  who  have  participated  in  Head  Start  will  be 
superior  in  reading  readiness." 


9.    Miss  Brown  replied,  "I  don' t  believe  that.    Our  principal  says  the 
Head  Start  program  did  no  one  any  good.    Therefore,  I  expect 
the  Head  Start  children  to  be  no  better  on  reading  readiness  than 
the  other  students . " 


10.    Mrs.  Carter  said,  "Let  us  examine  and  measure  the  children  on  reading 
readiness  in  every  way  we  can  to  determine  whether  the  Head  Start  children 
do  better  or  not 


11.    One  difference  between  the  experimental  and  desjcriptive  methods 
in  research  has  to  do  x^7ith  the 

a.  setting  of  the  study* 

b.  control  of  variables. 

c.  number  of  subjects. 
dV  recording  of  data. 

e.     time  span  of  the  study. 
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12*    The  most  important  characteristic  of  the  experimental  method  is 

a,  the  repetition  of  observations, 

b,  -  the  control  and  systematic  variation  of  the  conditions  of 

observation. 

c,  the  making  of  exact  measurements- 

d,  the  concept  of  correlation. 

13.    An  investigator  was  interested  in  studying  teacher  success.  He 
selected  two  groups  of  teachers:     one  that  was  highly  successful 
and  one  that  was  not  too  successful.     He  found  that  the  more  successful 
teachers  tend  to  be  more  out-going*  more  independent,  and  more 
interested  in  people  than  less  successful  teachers.     The  type  of 
design  used  in  this  study  was 

a.  experimental. 

b.  ex- pja^.  fcaciiL. 

c.  longitudinal. 

d.  correlational. 

e.  cross-sectional. 

Midland  Central  School  District  has  recently  adopted  the  "mew  math" 
into  its  program  at  the  7th  grade  level.     It  is  now  interested  in  a 
study  of  6th  grade  students  in  order  to  evaluate  the  effects  of  the  new 
method.     One  hundred  students  were  included  in  the  study,  25  from  each  of 
the  four  elementary  schools  in  the  district-    The  25  students  from  each  school 
were  identified  and  included  in  the  study  in  such  a  way  that  all  students 
in  the  6th  grade  in  each  school  had  an  equal  opportunity  to  appear  in  the 
25  from  each  school.    The  tv^i^.al  of  100  students  were  then  placed  into  three 
testing  groups  by  a  method  which  assured  that  there  x<rould  probably  be  no  ; 
differences  between  these  three  groups  in  terms  of  extraneous  variables. 
This  was  done  so  that  three  different  forms  of  a  standardized  achievement 
test  could  be  used  and  compared.    The  students  were  then  administered  the 
tests.    The  results  were  analyzed  and  conclusions  dra^^  concerning  the 
success  of  the  new  method  with  6th  graders  In  the  school  district. 

14.  Random  selection  was  involved  in 

a.  placing  students  in  each  of  the  three  testing  groups. 

b.  administering  tests  to  the  three  testing  groups. 

c»     identifying  25  students  from  the  6th  grade  in  each  school, 
d.     deciding  to  use  modern  math  as  an  instructional  method . 
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Suppose  a  researcher  wanted  to  determine  the  effects  on  the  attitudes 
of  citizens  toward  centralized  government  of  the  electrical 
blackout  which  occurred  November  9,  1965,  in  Northeastern.  United 
States,     He  hypothesized  that  those  people  living  within  the 
blackout  area  would  be  more  opposed  to  Federal  control ,  sina' 
a  Federally  afeitiated  grid  system  allowed  the  blackout  to  speesd 
from  its  origftti  and  since  these  people  were  actually  affecteii 
by  the  blackcp^t.    He  selected  a  stasrdardized  >  objective  instrument, 
to  measure  attitudes  toward  centraiM^zed  versus  local  control - 
Assume  that  norms  on  this  instrument:  were  available  for  the  natiOTL 
as  a  whole,  Che  blackout  region  and;  the  region  outside  the  blackout: 
area.    He  selected  ,05  per  cent  random  samples  from  the  citizens 
21  years  of  age  or  older  living  in  the  blackout  area  and  from 
those  living  outside  the  blackout  area.    He  administered  and 
scored  the  instrument  and  tested  the  significance  of  the  difference 
between  average  scores  for  the  two  groups.    He  found  a  marked 
difference:     the  persons  living  in  the  blackout  zone  were  much  more 
opposed  to  Federal  control-    He  thus  concluded  that  the  blackout 
did  in  fact  affect  attitxides  toward  centralized  government  for  those 
who  actually  experienced  the  blackout. 


15^    ^'Jhat  major  type  of  inquiry  best  characterizes  the  design  of  this  study? 

a,  ex  post  facto  research 

b,  true  experimental  research 
c*     sample  survey  research 

d,    attitudinal  inventory  research 

16.    \*fhich  of  the  following  problems  would  most  require  historical  research? 

•      a.;    trends  in  patterns  of  financing  Georgia  public  schools 

b.  current  concepts  of  'discipline''  of  public  high  school  principals 
in  Georgia 

c.  effects  of  '^social  acceptance"  on  tendencies  to  "withdraw" 

d.  the  developmental  needs  of  high  school  youth  in  Clarke  County^ 
Georgia .  .'■ 

17«    T^ich  one  of  the  following  is  an  appropriate  step  to  take  in  the 

external  criticism  of  an  original  document  used  in  historical  research? 


a.  determining  who  actually  wrote  the  document . 

b-  establishing  the  author's  possible  motives  for  writing  the  document. 

c.  making  a  study  to  determine  the  author's  meanings. 

d.  analyzing  the  author's  possible  bias. 
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18.    One  of  the  weaknesses  of  the  historical  method  is  that  the  researcher 
has  available  as  data  ^'selected  conceptiioas  of  reality  rather  than 
reality  itself  J'    All  of  the  followin^i  are  implications  of  this  ' 
weakness  EXG^EPT 

a.  imporitant  facets  of  historical  events  may  never  be  accessible 
to  tbe  researcher. 

b.  the  descriptions  of  reality  available  to  the  researcher  may 
be  inaccurate  or  in  error . 

c.  the  findings  of  historical  research  must  be  considered 
undepCTidable 

d.  extenral  and  internal  criticisms  of  the  data  are  crucial 
to  thB  success  of  the  research- 

L9.    When  reading  a  research  report  in  a  respectable  journal,  we  should 

a.  accept  the  conclusions  without  question/ 

b.  accept  the  conclusions  without  question  only  if  the  author 
is  a  well-known  authority. 

c.  accept  those  conclusions  that  agree  with  our  opinion. 

d.  reject  all  conclusions  if  they  are  based  on  an  inductive 
argument, 

e.  evaluate  the  conclusions  according  to  the  correctness  of 
the  research  methodology. 

to*     The  acceptance  of  experimentation  in  education  has  been,  according 
to  Campbell  and  Stanley, 

a.     increasing  steadily  over  the  past  six  decades. 

b*     sporadic  overall  but  highest  in  the  1920 'sand  lowest  in 

the  mid~1930's/  . 
c,     decreasing  steadily  over  the  past  six  decades* 
d*     at  a  constantly  high  level  over  the  past  four  decades. 

11.  A  journal  in  which  appear  brief  summaries  of  the  substance  of  ^ 
primary  research  completed  within  a 'selected  broad  division  of 
education  over  a  given  period  of  time  (e.g.,  three  years)  is  the 

a.  Review  of  Educational  Research^, 

b.  Educational  Abstracts, 
Journal  of  Educational  Research. 

d  .     Educational  Resear  ch  Bulletin  * 

12.  The  most  recent  data  regarding  dependable  knowledge  in  a  given 
educational  field  are  likely  to  be  found  in 

a.  The  Encyclopedia  of  Educational  Research  » 

b.  A.E.R.A.  Handbook  of  Research  on  Teaching , 

c .  Yearbooks  of  the  National  Society  for  the  Study  of  Education. 

d.  current  research  periodicals . 
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22^.    In  vfcich  one  of  the  following  womid  one  find  a  critical  reviev;  of 
the  f^qwa  Testt  of  Basic  Skills? 

Mental  Measurements?  Yearbook 
b*     Encyclopedia  of  Educational  Research 
^    Buros '  Tests  in  Pr±nt 

fl-.     Educational  and  Ps:srcholo:gical  Measurements 
Review  of  Educational  Research 

33£6£r  f ollGssTing  research  resources  apply  to  items  24  -  27  below: 


1 •  Handbook  of  Research. on  Teaching 

2.  Encyclopedia  of  Educational  Research 

3.  Review  of  Educational  Research 

4.  NEA  Journal 

5 .  Readers  Guide 

6 .  Psychological  Abstracts 

7.  Psychological  Bulletin 

8 .  Tests  in  Print 

9  *  Mental  Measurements  Yearbook 

10.  American  Educational  Research  Journal 


a)  1 , 


b)   ;  2, 


c)  3, 


d)  8, 


e)  9 


25. 


To  obtain  a  critical  review  of  the  Lorge-Thomdike  Intelligence  Test 


a)  2, 


b)  3, 


c)  7, 


d)  9, 


e)  10 


26. 


A  general  and  complete  suinmary  of  research  within  the  last  three 
years  on  "Educational  and  Psychological  Testing'': 


a)  1, 


b)  2, 


c):  3, 


d)  6, 


e)  9 


27. 


A  high  quality  publication  of  original  research; 


a)     3,        b)    6,         c)     7,         d)  IC 
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II*     Steps  in  the  Process  of  Research 

28.  In  conducting  research ,  which  of  the  following  steps  should  generally 
be  taken  first? 

a.  collection  of  data 

b.  compilation  of  a  bibliography  of  similar  researches 

c.  formulation  of  a  working  hypothesis 

d.  careful  formulation  of  the  problein  to  be  solved 

29.  A  hypothesis  is 

a^  a  general  statement  which  has  been  proved. 

b.  a  general  statement  not  sub ject  to  proof . 

c.  a  statement  temporarily  accepted  as  true. 

d.  a  statement  considered  as  false  until  proved  true. 

30.  The  lowest  level  in  the  hierarchy  of  certainty  of  scientific  knowledge 
is  represented  by  the 

a.  hypothesis. 

b .  theory . 

c.  law. 

d.  principle. 

31»    A  substantive  or  research  hypothesis 

a.  is  directly  testable. 

b.  has  to  be  translated  into  operational  and  experimental  terms . 
c»     is  dependent  on  the  statistical  hypothesis. 

d.     predicts  how  the  analysis  of  quantitative  data  will  be 
•  ■  resolved. 

32.  The  first  step  in  the  empirical  testing  of  a  research  hypothesis  is 

a,  collecting  relevant  data. 

b,  deducing  consequences  that  can  be  observed. 

c,  selecting  or  developing  tests  or  observational  systems  that 
will  provide  data.; 

33.  When  a  general  problem  of  interest  to  an  individual  arises  and  he 
makes  the  decision  to  undertake  a  research  project  towards  its 
solution,  he  must  first  identify 

a.  the  studies  he  will  pattern  his  project  after, 

vb .  the  tests  available  to  solve  the  problem. 

c.  the  variables  inherent  in  the  problem. 

d.  the  inference  pattern  related  to  the  problem* 
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34. 


The  scientist  affords  the  facts  a  chance  to  prove  or  disprove 
something  by  hypothesizing  in  advance . 


b. 


True 
False 


35. 


Hypotheses  are  statements  about  relations  between  variables. 


a. 
b- 


True 
False 


In  items  36-40  evaluate  the  adequacy  of  each  statement  as  a  scientific 
hypothesis  by  using  the  following  key : 

a.  Adequate  hypothesis 

b.  Inadequate,  less  than  two  variables 

c.  Inadequate,  no  statement  of  relationship 

d.  Inadequate,  not  testable 

36.     If  programmed  text  materials  are  used  to  teach  arithmetic  drill 

materials,  then  mastery  of  the  materials  (90%  or  better  on  a  critetion 
test)  will  follow. 


37,     If  subjects  vary  in  their  level  of  anxiety  (as  measured  by  the  M.A.S,), 
then  they  x^ill  differ  in  their  achievement  on  a  timed  spelling  test. 


38.     Frustration  (produced  by  preventing  children  from  reaching  soluttions 
to  problems)  produces  agression. 


39.     If  people  are  prejudiced,,  then  they  identify  minority  group  members 
by  their  faces  more  readily  than  do  unprejudiced  people. 


40*     Group  therapy  (as  opposed  to  non-group  therapy)  will  result  in 
individuals  with  more  adequate  value  systems . 
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The  report  of  a  study  began  x^rith  this  statement:     "The  purpose  of 
this  study  is  to  determine  the  differences  between  'under  achieving' 
and   'normally  achieving'  students  in  the  seventh  and  eighth  grades 
with  respect  to  variables  in  personality,  aptitude,  study  habits, 
and  home  environment  J'     Is  the  statement  a  hypothesis? 

a.  No,  because  the  statement  is  not  posed  in  the  form  of  a 
question* 

b.  No,  because  it  does  not  propose  a  relationship  to  be  tested. 

c.  Yes,  because  it  will  lead  to  the  testing  of  the  significance 
of  differences. 

d.  Yes,  because  it  does  refer  to  the  main  purpose. 

Which  of  the  following  is  a  necessary  characteristic  of  an  adequately 
defined  problem? 

a.  It  is  based  on  previous  research. 

b.  It  is  developed  from  clearly  defined  objectives- 

c.  It  is  based  on  the  analysis  of  data  already  available. 

d.  It  leads  logically  to  the  listing  of  testable  hypotheses. 

The  most  important  criterion  for  evaluating  the  procedures  ^section 
of  a  research  study  is:     Has  the : author  explained  his  procedures 
in  such  a  way  that  will  enable  the  reader  to  the  study? 

a.  criticize 

b.  comprehend 

c.  statistically  analyze 

d .  repeat 

e.  deduce  consequences  of 

In  selecting  a  group  for  a  study,  the  initial  step  is  to 

a.  identify  all  the  units  which  compose  the  total  population. 

b.  use  a  table  of  random  numbers  to  help  provide  anmnbiased 
sample .  ,  ' 

c.  provide  for  double-sampling  techniques  to  insure  a  represen- 
tative sample  for  study  * 

d.  sample  every  odd  number  item  and  compare  the  obtained  results 
with  a  sample  of  every  even  numbered  item. 

All  of  the  following  are  dangers  to  be  watched  for  and  weaknesses 
to  be  corrected  in  the  collection  and  analysis  of  data  EXCEPT  which 
■  one?;  •  ■ 

a.  Are  the  data  adequate? 

b.  Are  the  data  carefully  sifted  and  consi^  for  accuracy  and 
.  au^thentici^ty? 

c.  Do  the  datna  substantiate  the  hypothesis? 

d.  Does::Lthe  evidence  really  mean  wh^  mean— are 
the  conclusions  sound? 


-10- 


46.  The  statistical  methods  to  be  used  in  a  study  should  be  considered 

a.  before  the  hypotheses  have  been  formulated. 

b.  as  the  study  is  being  designed. 

c.  after  the  data  have  been  collected, 
d*  after  the  data  have  been  tabulated. 

47.  A  null  hypothesis  in  a  comparative  experiment  is 

a.     always  false. 

b*     always  the  same  as  the  research  hypothesis. 

c.  a  statistical  hypothesis  that  assumes  that  the  subjects 

in  two  treatment  groups  were  equal  before  treatment  began, 

d.  a  statistical  hypothesis  that  assumes  there  is  a  difference 
among  the  effects  of  treatments . 

e.  a  statistical  hypothesis  that  assumes  there  is  no  difference 
among  the  effects  of  treatments. 

48.  Defining  "learn  more"  by  a  score  on  the  Iowa  Test  of  Basic  Skills 
is  a  (an) 

a.  measured  operation  definition. 

b .  experimental  operational  definition, 

c.  constituted  definition. 

d.  synonym  definition. 

Read  and  examine  the  following  statement •     For  each  term  on  the  left, 
select  the  phrase  on  the  right  (taken  from  the  statement)  which 
correctly  ■illustrates  each  term.     Each  phrase  may  be  used  more  than  once. 

'*What  effects  do  socio--economic  background  factors  have  on  students* 
ability  in  visual  perception  as  measured  by  performance  in  the  Renshaw 
Visual  Acuity  Test?" 

A9.         dependent  variable  a)  socio-economic  background  factors 
answer: 

50.  construct /intervening  variable  b)  ability  in  visual  perception 
■  answer 

51.  operational  definition  c)  performance  in  the  Renshaw  Visual 
answer  Acuity  Test 

52.  indepBndent  variable 
answer 
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III.     Experimental  Research 

53*      According  to  Kerlinger,  one  basic  purpose  of  research  design  is 

a.  to  provide  research  questions, 
to  suggest  research  problems . 

c.  to  control  variance. 

d.  to  control  independent  variables. 

54.  The  main  technical  function  of  research  design  in  education  is  to 

a*  control  extraneous  systematic  variance, 

b.  minimize  error  variance. 

c.  maximize  systematic  variance. 

d.  (b)  and  (c) 

e*     all  of  the  above 

A  sociologist  surveyed,  by  means  of  a  mail  questionnaire,  the  attitudes 
of  persons  who  managed  a  certain  group  of  hotels  and  restaurants  as 
to  whether  they  would  accept  Indians  as  guests  or  customers.     He  then 
arranged  for  an  Indian  couple  to  visit  these  hotels  and  restaurants, 
and  subsequently  learned  from  the  couple  which  establishments  had 
actually  served  them.     He  found  that  of  the  establishments  which  had 
served  ^the  Indian  couple,  over  90  percent  had  previously  stated  they 
would  not  serve  Indians. 

55.  Expressed  attitude  toward  a  course  of  action  is  not  necessarily 
a  reliable  indicator  of  behavior. 

a*     This  conclusion  is  warranted  by  the  study. 

b.     This  conclusion  is  not  warranted  by  the  study. 

\    56.      Surveys  measuring  expressed  attitudes  contribute  little  to  the 
understanding  of  what  people  will  do  in  everyday  practice. 

a*     This  conclusion  is  warranted  by  the  study. 

b.     This  conclusion  is  not  warranted  by  the  study. 

57.  Campbell  and  Stanley  introduce  McCallfs  work  as 

a.  an  example  of  the  best  current,  thought  in  experimental  design* 

b.  a  natural  follow--up  for  the  x>^ork  dbne  earlier  ty  Fisher. 

c.  an  early  yet  insightful  statement  itof  fundamental  design 
techniques  and  methodologies r 

d .  an  example  of  heavy  reliance  on  designs  of  the  pre-experimental 
■.■type.";;  ^  ,  ■  '  • 

58.  A  sample  is  considered  to  be  random  if  every  element  of  the  population 
has  an  equal  and  independent  chance  of  bei^ag  included  in  the  ^s^ 

Q  a.  True 

ERjC  ^-     False  . 
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59.     Statistics  is  to  parameter  as 

a.  population  is  to  sample. 

b.  sample  is  to  population. 

c .  mean  is  to  variance . 

d.  variance  is  to  mean. 


60. 


61, 


vThen  a  researcher  states  that  a  result  is  significant,  he  means 

a.  the  effect  is  a  practically  important  one. 

b.  the  gain  scores  are  not  correlated. 

c.  the  result  is  unlikely  to  be  a  chance  occurrence. 

d.  the  sample  values  are  different. 

The  results  of  an  experiment  are  said  to  be  confounded  when 


a.  they  contradict  previous  research  findings. 

b.  they  are  contrary  to  what  was  predicted, 
c*     they  are  unexpected. 

d.     the  effects  of  the  variables  cannot  be  unambiguously  evaluated, 
the  variables  interact  v/ith  each  other. 

Which  of  these  terms  least  belongs  with  the  others? 


a.  experimental  design 

b.  randomization 
c»  correlation 
d»  manipulation 
e .  control 


^3*     Of  the  following,  the  most  important  determinant  of  the  extent 

to  which  the  results  of  a  research  study  can  be  generalized  is  the 

^       a.     nature  of  the  sample  used  in  the  study. 

b.  adequacy  of  the  instruments  used  to  collect  data. 

c.  degree  to  which  the  effects  of  extraneous  variables  have 
been  controlled. 

d.  ease  with  which  the  research  design  can  L»e  replicated. 

The  major  function  of  the  control  grc'dp-  in  an  experimental  study 
is  tOy.measure  the  effect  of 


a.     differences  in  difficulty  of  pre-  and  post-  tests, 
external  factors  upon  the  dependent  variable. 

c.  interaction  of  group  characteristics  and  experimental  * 
treatments 

d.  :  administration  of  a  pre-test  prior  to  application  of  the 

experimental  treatment. v 
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.owing  to  answer  questions  65-69 - 

I  large  high  school  thinks   that  ballet   training  will 
batting  average  of  his  baseball   team .     He  decides  to 
le  team  take  six  weeks   of  ballet  training  before   the  base- 
begins  while  the  other  half  does  not   take  such   training - 
I  compare   the  season  batting  averages  of  group  A,  those 
training,    and  group  those  without  ballet   training,  by 

le  mean  of   group  A  with   the  mean  of  group 

idy  would  be  classified  as 

I  school  survey 

post  facto  or  casual— comparative  study. 
i  correlational  study • 
I   trend  study, 
ixperimental  research . 

:  to  test  his  hypothesis  5  the  coach  should  have  which  group 
-let  training? 

:he  best  players  . 
:he  poorest  players 
I  randomly  selected  group 
:hose  who  volunteer 

;ood  players  who  show  poor  coordination 

:rol  group  v7ould  be  those  who  . 

ike  ballet  training* 
lo  not  take  ballet  training  • 
-ry  cut  for  the  baseball  team, 
lave  high  batting  averages . 

ire  willing  to  cooperate  in  the  experiment 

ipendent  variable  is 

>allet  training 
>atting  average 

runs  batted   in  - 
:he  size  of   the  school 

:lie  grades   players  make  in  ballet  school 

idy  would  be  classified  • 

Dne  group  pre-tes t/pos t-tes t 
iiontrol  group  pre-test /post- test 
::ontrol  group  post-test  only 
Solomon  four  group  design 
::or relational  study  • 
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Suppose  the  group  with  ballet  training  completes  the  season  with 
a  higher  mean  batting  average  than  the  group- without  ballet 
training  and  statistical  tests  indicate  that  this  difference 
is  highly -unlikely  to  be  a  functi6n  of  chance.    Assuming  that 
no  problems  of  internal  validity  are  present,  the  coach  should: 


a. 
b. 
c. 


reject  the  null  hypothesis. 

reject  the  experimental  hypothesis. 

declare  the  experimental  hypothesis  is  proven. 


71-    The  external  validity  question  in  this  study  would  be; 


a. 

b. 

c , 
d. 


Was  ballet  training  really  responsible  for  the  differences 
in  batting  average? 

Was  the  coach  unethical  in  making  baseball  players  take 
ballet  training? 

Is  ballet  training  worth  the  expense  it  involves? 

Would  ballet  training  improve  the  batting  perform^tnce  of 

other  teams? 


72.    Hawthorne  effect  would  be  most  likely  to  increase  the  scores  of 

a.  the  best  players. 

b.  the  poorest  pla3^ers. 

c.  the  players  with  ballet  training. 

d*     the  players  without  ballet  training, 
e.    all  players . 


73. 


7 


74, 


One  of  the  main  reasons  for  working  with  samples  rather  than  populations 
is  that 


b- 
c. 
d. 


populations,  being  finite ^  cannot  be  discussed  in  terms  of 
intervals . 

the  experimenter,  because  of  practical  reasons ,  rarely  can 
locate  or  test  the  population. 

statistics  are  appropriate  to  use  with  samples  but  not 
appropriate  to  use  with  populations. 

population  measures  are  biased,  whereas  the  repeated  use  of 
samples  controls  bias. 


I^hich  of  the  following  precedures  would  yield  the  most  appropriate 
data  for  studying  the  relation  between  intelligence  and  achievement? 

a.  administering  an  achievement  test  and  an  intelligence 
test  to  one  sample  of  subjects. 

b.  administering  an  achievement  test  to  one  sample  of  subjects 
and  an  intelligence  test  to  another  sample  of  subjects. 

c.  admir.istering  an  achievement  test  to  a  sample  of  subjects, 
all  of  whom  have  an  I.Q.  of  100.  .  .] 

d.  administering  an  achievement  test  to  2  samples  of  subjects 
and  an  intelligence  test  to  2  different  samples  of  subjects . 
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75.     The  entire  collection  of  elemsnts  or  measurements  about  which  an 
:  inference  is  to  be  made  is 


a.  an  experimental  design. 

b.  an  interval  est iinate, 

c .  a  population o 

d .  a  sample . 

76.  Experimental  and  control  groups  are  f^re^uently  equated  by  matching 
'       pairs  of  subjects  on  one  or  more  variables   (e.g.,  intelligence) 

and  assigning  one  member  of  each  matched  pair  to  the  experimental 
group  and  the  other  member  to  the  control  group .     What  factor  is  the 
most  crucial  in  determining  the  effectiveness  of  this  procedure 
for  developing  equivalent  groups? 

a.  whether  the  total  group  of  subjects  is  a  random  sample 
from  the  population  of  interest 

b.  whether  equal  numbers  of  males  and  females  were  assigned 
randomly  to  treatment  or  control  groups 

c.  whether  the  matching  variable (s)  are  exclusively  and  highly 
related  to  the  dependent  variable  of  interest 

d/    xdiether  the  study  is  experimental  or  ex  post  facto 

77.  rt?;raraeter  is  to  population  as 

a,  statistic  is  to  sample. 

b,  sample  space  is  to  frame  of  reference. 

c,  statistic  is  to  probability , 

d/    non"-parametric  is  to  parametric. 

e.  statistic  is  to  measurement. 

78.  If  a  and  3  are  characteristics,  of  population  P,  and  if  a  and  b 
are  estimates  of  a  and  3  on  the  basis  of  a  sample,  then 

a*  a  and  3  are  statistics  and,  a  and  b  are  parameters. 

b.  a  and  3  are  parameters  and,  a  and  b  are  statistics/ 

c.  ot  ,  3 ,  a,  b ,  are  statistics . 

d.  a,   3,  a>  b  j  are  parameters . 

79.  Which  of  the  following  terms  best  fits  the  following  definition? 
'*A  conclusion  about  a  population,  made  on  the  ba.sis  of  what  has 
happened  in  a  sample  of  observations ,  the  conclujion  being  drawn 
with  a  calculable  risk  of  being  in  error*V  is  a(n) 

a.  population  parameter, 

b.  random  selection. 

c.  statistical  inference. 

d.  unbiased  estimate. 
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Suppose  youwish  to  compare  experimentally  the  effects  of  two  drugs 
on  reaction  time.     You  randomly  select  30  patients  from  each  of  two 
hospitals.     You  administer  treatment  1  to  the  patients  in  hospital  A 
and  treatment  2  to  the  patients  in  hospital  B.     You  then  test  each 
subject  individually  on  a  single  apparatus  for  measuring  reaction 
time,  compute  the  means  and  standard  deviations  for  the  two  groups , 
and  calculate  a  two  tailed  t  test  using  the  *10  level  of  significance. 

80.  VJhat  source  of  error  is  most  likely  to  result  from  this  design? 

a*.     Initial  differences  between  hospitals  could  obscure  differences 
betx^een  the  effects  of  the  drugs. 

b.  Using  only  one  testing  apparatus  prevents  generalization. 

c.  Comparing  two  treatments  yields  too  little  information; 
instead,  each  should  have  been  compared  with  a  control. 

d.  The  effects  of  one  drug  may  Interact  V7ith  the  effects  of 
the  other . 

81.  Is  the  statistical  analysis  appropriate? 

a.  Yes,  the  t  test  will  yield  a  valid  estimate  of  the  statistical 
difference  between  the  test  results  for  these  samples. 

b.  Yes ,  the  t  test  will' yield  the  information  necessary  for  ; 
determining  which  drug  is  superior. 

c.  No,  a  one  tailed  test  is  required. 

d.  No,   the  .10  level  of  significance  is  too  large. 

82.  Which  of  the  following  correctly  states  the  extent  of  permissible 
generalization  in  the  formation  of  a  statistical  inference?  The 
generalization  pertains  to  all  individuals 

a.  possessing  characteristics  similar  to  those  used  in  the  study.  • 

b.  actually  used  in  the  study. 

c.  suggested  by  a  carefully  developed  analogy. 

d.  from  the  population  sampled. 

..83.     Characteristics  of  persons  or  things  which  can  assume  different 
values  are  called 

a.  distributions . 

b .  observations . 

c .  data. 

d •     variables . 

84.     In  an  experiment  to  investigate  the  extent  to  which  learning  changes 
systematically  with  increasing  age,  the  independent  variable  would  be 

O  a.     leng th  of  the  list  to  be  learned . 

ERJC  b .     age  levels  of  those  tested . 

c.     change  from  trial  and  error  to  insightful  learning . 


85.  The  handedness  of  a  person  is  a  (cin) 

a*  continuous  variable, 

b'.  discrete  variable.  ! 

c.  independent  variable. 

d.  dependent  variable. 

e.  intervening  variable • 

86.  In  an  experiment  to  determine  the  effect  of  the  tempo  of  music  on 
workers'  productivity  in  a  factory  setting ,  the  dependent  variable 
is  the 

a.  tempo  of  the  music. 

b.  factory  setting. 

c.  number  of  workers . 

d.  workers '  productivity . 

87.  Which  term  least  belongs  with  the  others? 

a.  dependent 

b .  ordinal 

c .  response 
d*  criterion 

88.  A  dependent  variable  is  generally  a  classification  variable  which 
may  be  manipulated  or  controlled  in  an  exper^iment. 

a.  True 

b.  False 

89.  The  dependent  variable  is  the  treatment  variable  which  the  investigator 
varies  for  purposes  of  controlling  the  results. 

;   a.  -  True 

■  b.^  -.False';.  ,  ■.■„•■■■' 

90.  Students  taught  first  aid  by  programmed  instruction  will  achieve 
at  a  higher  level  than  those  taught  first  aid  by  the  traditional 

.    method.     In  the  above  hypothesis  the  independent  variable  is 

a.  students . 

b.  level  of  achievement. 

c.  programmed  instruction. 

d.  method  of  instruction. 

e.  first  aid  test  scores. 
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E  plans  an  experiTnent  in  which  60  children  are  to  be  randomly  assigned 
to  four  treatment  groups :     extrinsic  rex-/ard   (candy )  and  extrinsic 
punishment   (loss  of  recess) ;  extrinsic  reward  only ;   extrinsic  punishment 
only;  and  neither  extrinsic  reward  nor  punishment .     Before  the  study 
begins,  E  gives  an  IQ  test  to  all  the  pupils   to  use  in  the  statistical 
analysis,  but  not  to  be. used  in  assigning  pupils  :to  groups .     To  determine 
the  effect  of   the  treatments ,   a  geography  test  is  given. 

91.      In  this  experiment,   the  assigned  independent  variable  is 

a.  treatment • 

b,  IQ. 

c,  geography  test  score* 

d.  There  is  none  in  this  experiment. 

92/    In  this  experiment ,   the  active  independent  variable  is 

a.  treatment, 
■b-  ■  IQ. 

c.     geography  test  score, 
d •     There  is  none  in  this  experiment . 

E  plans  an  experiment  to  determine  if  use  of  certain  specified  amounts 
of  a  drug  will  Increase  ti^e  IQ  scores  differentially  for  male  and  female 
pupils .     He  uses  fifth  graders  for  the  study* 

93.  In  this  experiment,   IQ  serves  as 

a»  an  active  independent  variable. 

b.  an  assigned  independent  variable . 

c.  a  dependent  variable . 
d •  Both  A  and  B  are  correct . 

94.  In  this  experiment ,  sex  serves  as 

a.  an  active  independent  variable • 

b.  an  assigned  independent  variable. 

c.  a  dependent  variable . 

d.  None  of  the  above  are  correct, 

95.  In  this  experiment ,   the  drug  serves  as 

a.  an  active  independent  variable. 

b.  as  assigned  independent  variable 

c.  a  dependent  variable. 

d.  None  of  the  above  are  correct. 

96.  The  experimenter  usually  has  as  his  objective  the 

a.  ;   minimizing  treatment  or  experimental  variance. 

b.  maximizing  extraneous  variance. 
Q                  c.     ralnimizlng  error  variance, 

d •     None  of  the  above. 
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The  English  department  of  the -Sunbury  Junior  High  School  Is  concerned 
x.7ith  improving  the  reading  spea'd  and*  comprehension  of  7th  graders . 
They  have  decided  to  conduct  a  training  experiment  utilizing  the 
Tachitron  and  appropriate  sets  of  training  cards .     While  developing 
the  design  for  the  study ,   several  important  considerations  were  raised. 
First,   it  appeared  crucial  to  be  abie  to  determine  whether  or  not 
improvements  in  reading  skill  as  noted  through  scores  on  a  reading 
achievement  test  would  in  fact  be  due  to  the  training.     Second,   it  was 
also  realized  that  in  order  to  avoid  mistakes  and  inconsistencies  in 
grading,   it  would  be  important  to  develop  an  objective  and  accurate 
method  of  scoring  the  test .     There  was  also  concern  about  the  effects 
of  other  factors  on  the  results  of   the  study.     It  was  noted,  first, 
that  many  of  the  students  came  from  disadvantaged  homes  and  had  limited 
language  experiences .     Other  students  had  come  from  a  grade  school 
system  that  employed  a  special  training  program  in  reading  *  The 
intelligence  range  of   the  students  was  quite  wide  and  might  also  be 
an  important  factor. 

In  the  above  study ,  experimental  variance  would  be  found  in 

a.  intelligence  test  scores. 

b.  scores  on  the  reading  achievement  test. 

c.  the  research  design  of  the  study. 

d .  the  method  of  scoring. 

Which  of  the  following  constitutes  a  type  of  error  variance  controlled 
in  this  study? 

a.  socio-economic  levels 

b.  the  initial  low  level  of  reading  ability 

c.  mistakes  and  inconsi^stencies  in  scoring  the  achievement  test 

d.  the  training  method 

An  example  of  an  extraneous  vnariable  in  the  study  would  be 

a.  previous  participatxqo  in  a  reading  training  program. 

b.  scores  on  the  readi^sg  ^achievement  test . 

c.  the  training  method • 

d.  mis takes  and  inconsistencies  in  scoring. 


97 . 


98. 


99. 


100,     "A  'true'   experimental  design  mus t  incorporate  pre-test (In  the 
Campbell  &  Stanley  sense)     TirLs  statement  is 

■  a.  true .  r, 

b .  false . 

c.  usually ,  but  not  always,  true. 

d .  usually ,  but  not  always ,   false . 
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lOIii     Whema  pre-testrsand  post^est  areaoxsed  on  all  subjects  In  the 
expEECimentalaitdicontroli  groups 

SL-     the  pre--test  may  i3a 

b .  the  E  knows  whether  it  is  x^easonable  to  assume  that  both 
groups  began  at  a  similar  point  or  not . 

c.  there  may  be  interaction  betx>7een  pre- test  and  treatment 
effects     -    ■  , 

d.  a  and  b  are  both  true, 

e.  a,  b,  and  c  are  all  true. 

If  Method  A  works  better  with  bright  (intelligent)   students  and  Method  B 
with  dull  students   (as  measured  by  the  ABC  test,)   and  the  pattern  is 
even  more  pronounced  for  boys  than  girls ,  we  know: 

Indicate  True  or  False  for  items  102-107 

102.  Girls  achieved  better  than  boys. 

■  a,    . True  ■ 

■  b •  False 

103 .  The  overall  girl  and  boy  means  differed  significantly. 

a.  True 

b .  False 

104*     There  was  a  sex-by-intelligence  interaction. 

a.  True 
bw  False 

ZL05»     There  was  a  method--^by-intea3Iigence  Irrreraction  * 

•  a'.     True  - 
b^  False 

106  •     Theres    ^as  a  method-by--inteIXigence— i^^ex  interaction 
a^  True 

b*..   j35alse  .■  .  _^ 

1X51 .     The  overall  means  fbr  Method!  A  and  B  differed  significantly. 

a.  True 

b .  False 

108.     In  the  above  factorial  design,  how  many  factors  are  there? 
a)     0,      b)     1,       c)     2,       d)     3,       e)     4  or  more 


ERIC 


-109.    How  many  dependent  variables  are  there:? 


a)     0,       b)     X,       c)     2,       d)     3,       e)     4  or  more 
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110.       T^ich  type  of  sampling  is  used  if  the  population  is  divided  into 
groups  by  certain  common  characteristics  and  random  sampling  is 
done  from  each  group? 


a .  clus  ter 

b.  systematic 
c\  stratified 
d*  proportional 
e,  double 

111.       A  researcher  draws  a  sample  by  selecting  every  10th  name  in  a 

telephone  directory.  Which  of  the  following  sampling  types  has 
he  used? 


a*  simple  random 

b  •  double 

c.  stratified 

d.  systematic 


112*       If  50  students  from  a  group  of  200  7th  termers  were  to  be  studied 
by  the  Rbrschach  technique,   the  sample   (50)  would  be  considered 
random  if  . 

a,  there  were  the  sameiinumber  of  boys  ;andrigirls  in  the  group 

■  ■    of  "5o: 

b.  every  boy  and  girl  in  the  7th  term  had  ^n  equal  opportunity 
J.  to  be  selected  . 

c»     two  students  were  selecteii^  whose  names  began  with  each  of 
the  25  ietters  o£  t^he  alpfeabet  except  X- 

d  .     every  IQ  level  (ex:     80-"85:,85-90 ,  90-95 ,  etc .  )  was  equal^^ 
represented  by  the  50  students. 

113.       Selecting  the  children  to  :be  involved  in  an  experiment  by  randomly 
identifying  the  schools  to  be  involved  represents 


a.  stratified  sampling* 

b.  double  sampling. 

c.  systematic  sampiiiag. 

d.  cluster  sampling:*. 


114.       Two  characterxstics  of  a:  random isample  are :     (1)  independence  of 
selections,    (2)   equal  probabilities  of  selection. 

lis.       If  a  sample  is  large  and  randomly  drawn,   then  it  is  quite  likely 
it  is  representative  of  the  population. 

a.  True  ^ 

b.  False 


±2^/      Random  sampling  guards  agaibnst  known  and  unknown  biases. 

a.  True 

b .  False 


117.  Separating  a  group  of  possible  subjects  into  categories  (e-g,  males 
and  females)  before  selecting  subjects  (by  use  of  a  table  of  rand^fain 
numbers)   to  receive  a  particular  treatment  is  known  as  drawing  a 


a.  stratified  random  pojDxilation . 

b*  simple  random  sample, 

c  •  unbiased  random  .  saraEKELe  . 

d.  stratified  random  sample. 


118.       ^flien  a  sample  has  been  drawn  at  random,  we  can  still  only  make 

probabilistic  statements  abrout  its  relationship  to  the  populatiom 
from  which  it  was  drawn, 

a.  True 

b.  False 


119 .       Selecting  a  sample  at  random  means  that 


a.  thei  sample  drawm  :^H^  def  initely  be  representatdive . 

b.  euevry  member  of  the^:-^  f  rom  has  annequal  and 
independent  opportuniLty  of  being  selected  • 

c .  Both  A  and  B  are  coinrect. 

d .  Neither  A  nor  B  is  crorrect . 


A  sample  selected  to  assure  that  certain  subgroups  in  the  populaiLLiun 
will  be  represented  in  the  sample  in  proportion  to  their  numbers 
in  the  population  itself  is  called  a 


:   a •  random  sample . 

b  •  s  tratifi^eS  sanipSLe« 

c  .  pilot  sample . 

d .  heterogeneous  sam^Ee . 

The  most  efficient  means  of  ruling  out  factors  such  as  maturatiojaail 
ievel,   ability ,   age ,   etc.,   as  rival  explanations  of  ^an  observed 
post-test  difference  between  an  experimental  and  a  control  group  3es 


a •     careful  matching  of  the  experimental  and  control  groups  rem 
all  possible  factors   (e.g. ,   sex,   IQ,   and  achievement ) . 

b.  a  time-series  quasi-experimental  design • 

c.  random  assignment  of  subjects  to  the  treatment  groups . 

d .  the-  administration  of  a  pre- test . 
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122.  An  investigator  wishes  to  evaluate  the  influence  of  three  methods 
of  teaching  reading:  the  basal  reader  method,  the  I .T .A.  method , 
and  the  individualized  reading  ins truction  method.  In  setting  up 
his  groups  for  study,  he  should  be  sure  to  = 

a/  assign  the  children  randomly  to  classes. 

b.  assign  teachers  randomly  to  classes . 

c .  assign  the  method  of  reading  instruction  randomly        classes . 

d.  all  of  the  above.  . 
e\  none  of  the  above. 

123.  In  a -large  high  school  the  25  students  who  scored  highest:  in  an 
English  pre- test  were  pii  class.     At  thei^^end  of 
one  semeSiter  they  were  given  another  English  achievement:  Itest . 
The  meanv^^ade  level  equivalents  were  compared  for  the  isnro  tests - 
This  xlsr^asESBxample  of 

a.  one>  group  npre-test/post-test  design. 

b.  rarrdomized;  control-group  pre-test/piost- test  design, 
c*.  randomized  Solomon  four  group  design- 

d.  randomized  control-group  post-test  only  design 

e.  f actoxial:  design . 

124.  The  '"halo"  effect  in  rating  refers  to 

a.  irtfluence  of  one r.rater  upon  anothex. 

by  tendency  to  rate  ia  person  higher  .when  you  know  Ihtfin  better . 

c.  sp^read  of  a  general  impression  of  a  person  to  thea.rating 
o  ofe:  specif  ic  characteristics - 

d.  tendency  to  make  ratings  too  high - 

e.  tendency  not  to  be  too  hard  in  rating  people  oneadoesn*t  like. 

125.  Subjects  performing  well:: merely  because  they  are  being  :c^served 
(and  not  meces sari ly  because  of  any  effect  of  treatmen:t:)i^;:^are; 
considered  to  be  under  the  inf  luence  of 

a .  the  Hawthorne  ef £ect . 

b  *  the  novelty  effect, 

c.  the  halo  effect . 

d»  none  of  the  above^ 

126..     The  internal  validity  of  an  experimental  design  is  concerned  with  . 
the  question : 

a.  to  what  extent  can  the  experimental  findings  be  generalized? 

b.  did  the  independent  variable  really  produce  a  chaise  in  the 
dependent  variable? 

c.  how  representative  is  the  setting  selected  for  the  experiment? 

d.  will  the  findings  provide  information  about  situations  in 
which  variations  of  the  independent  variable  are  present? 


127.  When  the  resuits  of  an  experiment::  are  true  only  under  certain  very 
special"  condi-tions  and  do  not  generalize  far  beyond  these  particular 
experimental:  conditions ,   the  expexciment  is  said  to  have 

a.  low^  5£mernal  validity. 
b»  io^  external  validity, 
c.     high  external  validity. 

128.  Which  oi:  the  following  is  ara^.  Internml  validity  question  in  experimental 
research? 

a.  Has  uina.turation  infiuenced;  Lttite  dependent  variable? 

b.  Was  -til e  sample  representaitfEve  of  the  population? 

c-     Has  :tl3ie  hypothesis  heenslisrtced  ^^i  form? 

d*     Hx)w^::w3:del.y  can  the  resulLts^coE  the  expieriment  be  generalized? 

e.     Arei  ttke  potential  results  mf  the  studjr  worth  the  cost? 

129.  Which  texmilileast  belongs  x^itE3iB  ±^ 

a^  ±n1iE3mal  validity 

b .  ^esKtasCTtal  validity 

c.  :g;ei3Er3iLizability 
dv  ireEpi^^s^entativeness 

130.  "To  what:  popr.ulations ,  settings,  :^eatrment  variables,   and  measurement 
variables  can  this  ef fecit  be  gen5ei^3a:zed?"  might  most  appropri^ 

be  asked  linijrelation  to  the  coiarcep^t  of 

a«  criterion  validity. 

b.  interEnal  validity. 

c.  ex^tEiiiutal  validity. 

d.  camstrruct  validity. 

131 .  "Did  in  fact  the  experimental:;  make  a  difference  in  this 
specific  exp:erimental  ins tance£''iraight  most  appropriately  be  asked  in 
relation  :to  the  concept  of 

a.  criterion  validity. 

b.  internal  validity, 
c*  external  validity, 
d.  construct  validity. 

132.  In  evaluating  a  research  project,  what  are  the  two  most  important 
criteria? 

a.  internal  validity  and  external  validity • 

b.  representativeness  and  generalizability . 
practical  significance  and  statistical  significance. 

d.     Internal  consistency  and  reproducibility. 
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133.     A  pre-test  of  achievement  is  given  to  anrain^^lteGEi      group  of  40 

students  five  minutes  prior  to  an  hour '  s  fifn^^dnnEcitrion  in  addition. 
The  post-test  that  immediately  followed  th^UcsKlmuct ion  showed 
a  10  point  gain  from  the  pre~test.     It  is  ixariiszSiuded  t         the  instruct! 
was  effective  in  producing  a  gain.  Which^^^^ 

sources  of  invalidity  could  also  explain  t3te|'^!^e---test  to  post-test 
gain? ; 

a.  statistical  regression 

b.  interaction  of  selection  and  matuiriaa^js^ 

c.  instrumentation 
d*  testing 

134;;.     Professor  "X"  graded  initial  essay  exams  then  graded 

the  f  iual  essays   to  see  if  imp r ovement  IfeS  ^rris^r^nr-ced .     Which  of  these 
leas t  jeopardizes  internal  validity? 

a .  history 

b.  maturation 

c.  testing 

d.  ±nstrumentation 
e  *  regression 

135  ;     X-Jhich  would  probably  be  the  most  q t •?  oi i g  "tT^Hrr7OT^r?t~^n  internal  validity? 

a.  history 

b.  maturation 

c .  testing 

d .  instrumentation 

e.  regression 

136.  A  threat  to  internal  validity  that  may  occi!t^  ffemm  processes  within 
respondents  operating  as  a  function  of  the.  ]^33s^sgge  of  time  per  se 
(and  not  specific  to  particular  events)   is  iism^^ 

a.  maturation 

b.  instrumentation 

c.  regression 

d.  reactivity 

137.  When  groups  selected  for  their  extreme  scares  :t^^  used 
2lrL  an  experiment  ,   a  likely  source  of  invalidix^ty^  iim^  s 

a.  selection 

b.  maturation 

c .  regression 

d.  interaction  of  selection  and  treatment 
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138.  In  experimental  design,  when  comparisons  are  made  of  groups  which 
have  been  selected  on  the  basis  o£  their  extreme  scores ,   the  post- 
test  means  of  the  groups   tend  to  move  toward  the  means  of   the  entire 
population  from  which  the  extreme  groups  were  selected.  This 
factor  which  jeopardizes  validity  of  the  findings  is  termed 

a.  differential  selection 

b.  interaction  of   selection  and  X. 
c-      experimental  mortality 

d,     statistical  regression 

139.  Which  of  the   following  is  a  factor  jeopardizing  the  external 
validity  of  an  experiment? 

a.  statistical  regression 

b.  interaction  of  selection  biases  and  the  experimental  variable 

c.  instrumentation  . 

d .  history  - 

140.  Children  having  attended  nursery  school  were  compared  with  those 
having  no  such  experience  on  a  readiness  test  at   the  beginning  of 
Kindergarten  and  were   found  to  be  superior.     The  research  attributed 
this  superiority  to  the  children's  nursery-school  experience. 

The  greatest  threat  to  internal  validity  is 

a.  history 

b.  maturation 

c.  testing 

d.  selection 

e.  experimental  mortality 

141.  Pupils  in  the  remedial  reading  program  increased  in  problem  solving 
ability  during  the  course  of   the  year.     Attributing  the  cause  to 
the  change  of  the  program  overlooks  several  alternative  factors, 
the  most  significant  of  which  is  probably 

a.  selection 

b .  testing 

c.  experimental  mortality 

d.  regression 

142.  Students  with  low,  average  and  high  initial  self-esteem  were 
coiiiiseled.     The  low  group  gained  most  over  2  months.  Assuming 
the  counseling  5.iyas  more  ef fee tive  with  the  low  group  fails  to 
consider   (greatest  threat) 

a.  history  • 

b .  maturation 

c .  testing 
d  .  ins  trumentation 

e.  regression 
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IV.     Methods  of  Data  Collectiori 

143.  Annual  achievement  tests  which  are  parts  of  the  school  testing 
program,   illness  records ,   daily  assignments  and  other  routine 
characteristics  and  activities  of  the  school  are  considered  to 

■  be  ; 

a.  reactive  measures 

b,  nonreactive  measures 

144.  \^ich  of  the  following  is  most  likely  to  provide  the  best  description 
of  the  objectives  of  instruction  in  a  school  system? 


a.  teacher  ratings  of  the  students 

b.  standardized  aptitude  tests  . 

c.  teacher-TTiade  achievement  tests 

d.  standardised  achievement  tests 

145 <     In  which  way  are  teacher-made  tests  superior  to  standardized 

-  tests?--  -  ■■■  .     :  ■  ■■  ;  " 

a.  They  are  more  reliable  for  evaluating  differences  among 
very  poor  and  very  good  students. 

b.  They  provide  more  valid  measures  of  the  teachers'  specific 
objectives. 

c.  They  provide  a  better  measure  of  the  student's  grasp  of 
important  facts  and  principles. 

d.  They  are  simpler  to  administer  and  score. 


146.  T^ich  of  the  following  is  most  characteristic  of  skilled  as  opposed 
to  unskilled  observers  in  research? 

a.  making  generalized  descriptions  or  evaluations  rather  than 
noting  specific  incidents 

b.  interpreting  behavior  on  the  basis  of  limited  data  rather 
than  waiting  for  confirmation  by  subsequent  incidents 

c.  giving  the  child's  behavior  rather  than  the  personal  reaction 
of  the  observer 

d.  recording  primarily  dramatic  or  negative  incidents 

147.  Which  of  the  following  is  essential  to  skilled  observation  as 
a  research  technique  in  studying  a  pupil's  behavior? 


a.  recording  primarily  dramatic  behavior 

b.  recording  primarily  the  "whys"  of  the  behavior  observed 

c.  recording  primarily  the  "whats"  of  the  behavior 

d.  using  data  obtained  as  bases  for  generalizations  concerning 
the  behaviors  of  other  pupils 
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148.  According  to  Kerlinger,   sampling  of  behavior  by  the  observation 
method  can  be  considered  as  consisting  of  two  aspects: 

a,     person  sampling  and  event  sampling 
t> .     person  sampling  and  time  sampling 

event  sampling  and  time  sampling 
d.     criterion  sampling  and  person  sampling 

149.  Is  it  important  to  follow  up  those  persons  in  a  survey- sample  who 
have  failed  to  return  questionnaires  mailed  to  them?      (Assume  that 
the  direct-mail  method  is  appropriate.) 

a.  No,  because  a  certain  loss  must  be  expected. 

b.  No,  because  such  delayed  returns  would  be  of  doubtful  value. 

c.  Yes ,  because  the  sample  at  hand  may  be  biased. 

d.  Yes,  because  the  size  of  sample  should  be  as  large  as  possible 
150^     Interviews  and  questionnaires  as  a  data  collection  method 

a.  are  more  effective  than  observational  techniques . 

b.  reveal  only  information  the  subject  is  x-grilling  to  report. 

c.  cannot  be  considered  to  have  validity  - 

d.  provide  no  information  about  pas t  behavior . 

151.  Direct-mail  questionnaires  have  which  of  these  factors 
most  seriously  reducing  the  validity  of  their  "findings"? 

a.  history 

b.  regression 

c.  selection 

d .  ins  tr  umenta  tion 

e.  reactive  effects 

152.  Which  of  the  following  is  usually  the  most  serious  limitation  in 
the  use  of  the  mailed  questionnaire  in  descriptive  research? 

a.  defining  a  population 

b.  selecting  a  sample 

c .  obtaining  responses 

d .  analyzing  the  data 

153.  Which  one  is  not  a  characteristic  of  rating  scales? 

a.  difficult  to  construct 

b .  subject  to  bias 

c.  easy  to  administer 

d .  measures  remembered  behavior 
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154.  Which  of  the  following  techniques  assumes  that  people  will  reveal 
some  of  their  unconscious  feelings  and  attitudes? 

a.  sociometric  study 

b.  projective  tests 

c.  questionnaires 

d.  interviews 

Use  the  following  key  to  indicate  to  which  appraisal  instrument  each 
of  the  follovring  statements  best  applies- 

a.  Inventories 

b.  Rating  Scales 

c.  Tests 

d .  Sociometric  Techniques 

e.  Projective  Techniques 

155.  They  provide  data  on  the  interaction  among  group  members* 


156,  They  ask  an  individual  to  supply  information  concerning  his  typical 
behavior. 

157.  They  are  most  influenced  by  the  "error  of  leniency . 


158.  Osgood's  Semantic  Differential  technique  uses 

a.  many  concepts* 

b.  forced  dichotomies. 

c.  8  point  scales. 

d.  bipolar  adjectival  scales,. 

159 .  When  V:a  divide  children  into  English  speaking  and  Spanish  speaking 
groups,  we  are  using 

a.  nominal  scale . 

b .  ratio  scale. 

c.  ordinal  scale. 

d.  interval  scale . 

160.,  How  does  the  interval  scale  differ  from  the  ratio  scale? 

a.  The  interval  scale  has  no  absolute  Eero, 

b.  The  inteirvral  scale  has  equal  intervals* 

c.  The  medians  and  percentiles  can  be  computed. 

d.  The  interval  scale  can  be  compounded . 


ERIC 


Educational   Program  Management  Center 
Educational  Development  Faculty 
College  of  Education 
The  Ohio  State  University 
Columbus,  Ohio  43210 


CR ITER  I  ON-REFERENCED  EXAM  I  NAT  I  ON 
for 

PROGRAM/PROJECT  MANAGEMENT  COMPONENT 
July  6-10,  1970 


HEW  Institute 
on 

Planning  for  Decision  Making  in  the  '70's 
University  of  New  Hampshire 
Durham,  New  Hampshire 


This  examination  Is  not:  to  be  reproduced  in  whole  or 
part  without  express  permission  of  the  Educational 
Program  Management  Center,  College  of  Education, 
The  Ohio  State  University. 


ERLC 


(    )  K 


What  term  best  describes  the  primary  or  central  role  of  a 
project  manager? 


1 •  coord  i  nator 

2.  Information  generator 

3.  dec f s i on-ma ker 

4.  developer  of  products 

(     )      2.     What  statement  probably  best  characterizes  a  project? 

1.  Objectives  are  set  within  time,  cost,  and  performance 
specifications, 

2.  Usually  only  a  single  objective  is  involved. 

3.  Projects  are  usually  finite  in  character,  cofr  »iex  fn  nature^ 
consists  of  a  unique  series  of  tasks,  and  are  generally 

a  one-of-a-kind  activity, 

4.  Project  objectives  once  established  eliminate  the  need 
for  further  planning. 

(     )      3.     What  would  be  a  descriptive  title  for  an  activity  most  likely 
defined  as  a  project? 

K    The  Product  ion  of  a  Broadway  Play 

2.  A  Demographic  Study  of  Handicapped  Children  in 
Franki  in  County 

3.  The  Evolution  of  the  Theory  of  Relativity 
A,    The  U.  S.  Man  on  the  Moon  Program 

(     )      4.     Project  manager  authority  lines  are  least  clear  in  what  type  of 
project  organization? 

1 .  Vert  ica 1   1 ine 

2.  Hor i  zonta 1   n  ne 

3.  Separate  project  organization 

4.  Project  staff  approach 

(    )      5.    V/hat  is  generally  recognized  as  the  first  step  in  the  management 
process? 

!•  writing  a  proposal 

2.  personnel  employment  =  . 

3.  bidding  for  projects 

4.  establ I shing  objectives 

(    )      6.    What  Is  the  basic  function  of  a  inanagement  information  system  in 
a  project? 

1 .     communicate  the.  decisions  of  the  management  to  the 
performance  levels  of  the  project. 

2..  establ i sh  a  data  base  for  the  planni ng  stages  of  the 
proj  ect • 

3.     provide  the  manager  with  information  relevant  to  the  time, 

cost,  and  performance  factors. 
k.     provide  the  manager  with  progress  reports  from  a  11   1 evel s 

of  the  project . 
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(     )      7.     Optimum  program  is  a  combination  of  the  management  of  what 
three  principal  elements  of  information? 

1.  Planning,  Control  1  ing.  Organizing 

2.  Cost,  Tim^  Performance 

3.  Men,  Materlali  Missions 

4.  Ideas,  Actions,  Results 

(     )      8.    Management  by  exception  can  be  interpreted  to  mean  a  manager 
must  study  what  types  of  problems? 

1.  All   key  issues  brought  to  his  attention. 

2.  Key  deviations  between  the  plan  and  execution, 

3.  Those  presented  by  superiors  without  exceptic 
Probl ems  of  except  iona 1  vl s  ib  i 1 i ty- 

(     )      9.    What  is  the  chief  value  in  using  the  sys^cems  approach  to  define 
a  project? 

1.  it  establ i$hes  the  hierarchical  nature  of  the  objectives^ 

2.  it  allows  one  to  focus  on  limits  or  constraints  in  the 
environment. 

3.  it  permits  the  use  of  the  principles  of  disassembly  and 
assembl y. 

all  or  none  of  the  above. 

(    )     10.    What  do  we  call  the  process  of  breaking  down  a  system  into  inter- 
dependent parts? 

1 •  System  management 

2,  System  synthesis 

3.  System  analysis 
^.  System  sequence 

(    )     11.    What  major  subsystems  comprise  the  planning  system? 

1.  reports,  schedul ing,  project  plan,  management  actions 

2.  decision  implementation,  time  estimation,  scheduling, 
project  definition  1 

3.  project  plan,  project  def i n i tiori,  t Ime  est imat ion, 
scheduling,  cost/budget 

time  estimation,  reports,  project  plan,  decision 
implementation 

(    )     12,    What  are  the  three  major  subsystems  that  comprise  a  control 
system? 

l-     f'eports,  managements,  decision  implementation 

2.  time, -cost,  performance 

3.  project  definition,  network  system,  activities 
^.     tasks,  act! vitFes,  dummy  act ivities 
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(     )     13.    What  is  the  chief  product  of  the  project  definition  phase 
of  the  planning  system? 


1.  Personnel  Ass i gnments 

2.  Network  Diagram  ' 

3.  Work  Breakdown  Structure 
Time  Table  of  Events 

(     )     \k.    What  criterion  is  used  for  determining  the  number  of  levels 
necessary  to  define  a  project? 

1.  The  finest  detail  possible 

2.  Skeletal  breakdown 

3*    Necessary  deta i 1  based  on  project  manager's  Hesires  of 
wishes 

4.  Must  always  extend  to  five  levels  of  breakdown 

Using  the  following  terms,   identify  the  items  on  the  illustration  provided: 


1 .  Activity 

2.  Const ra  int 
3^  Dependency 

k.  Dummy  act  i  v  i  ty 


5»  Event 

6.  Interface 

7 .  Mi lestone 

8.  Network 


( 

) 

^5. 

Letter  A 

on  the  above  diagram 

represents 

a. 

( 

) 

16. 

Letter  B 

represents  a 

( 

) 

17. 

Letter  C 

represents  a 

( 

) 

18. 

Letter  D 

represents  a 

( 

) 

19. 

Letter  E 

represents  a 

( 

) 

20. 

The  term 

uti 1 ized  to  represent 

the  effect 

that 

F 

has 

on  G  is 

( 

) 

21 . 

The  term 

uti 1 i zed  to  represent 

the  effect 

that 

G 

has 

on  F  i  s 

f 

i 

22. 

The  entire  diagram  is  a 

(    )     23.     Time  est  imates  should  be  provided  by  what  person  on  the  project? 

1.  project  manager 

2.  project  wr  i  ter 

3.  the  person  performing  the  task 
an  efficiency  expert 

(    )     Zk.     How  is  scheduling  defined? 

K    action  which  adjusts  operations  to  predetermined  standards. 

2.  the  translation  of  the  plan  developed  from  the  previous 
steps  in  the  planning  process  into  a  time  table  showing 
the  specific  calendar  date  for  the  start  and  completion 
of  work. 

3-    a  statement  containing  the  major  goal  of  the  project^ 

but  also  includes  a  recognition  of  limits  and  constraints 
important  to  the  project. 

the  "organizational  unit  dedicated  to  the  attainment  of  a 
goal--general 1 y  the  successful  compl et ion  of  a  development 
product  on  t Imewi th i n  the  budget,  and  in  conformance  wi th 
performance  specification.** 

(    )     25.    Which  one  of  the  following  is  probably  the  single  most  important 
factor  in  schedul ing? 

1  .  t  ime 

2.  efficient  uti lizat ion  of  personnel  and  facilities 

3.  ava i lab i 1 i ty  of  resources 

4.  accounting  for  overestimate  abilities 

(     )     26.    Why  are  tradi t ional  budget i ng  and  account i ng  pract ices  generally 
not  c^ppl  icabl e  to  the  project  situation? 

1.  projects  are  non-cont  inuous  and  are  due  to  be  totally 
consumed. 

2.  the  trad i t iona ]  budgeta ry  mechan i sm  is  too  deta i I ed  and 
lacks  flexibility*  v 

3.  educational  budgets  are  based  upon  annual   or  biannual 
appropriations. 

4.  too  many  resource  and  personnel  al 1 ocat ions  are  on  a 
part-time  basis. 

(     )     27.    What  word  would  best  typify  the  management  approach  contrast 
to  the  traditional  approach  to  budgeting? 

1 .  I nputs 

2.  Outputs 

3.  Charge  number 

4.  Control 
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(    )    28.    What  is  management  control? 

!•    assuring  that  specific  tasks  are  carried  out  effectively 
and  efficiently. 

2.  collecting^  manipulating^  and  transmitting  information. 

3.  reporting  financial    i nformat ion  about  the  organization  to 
the  outside  world. 

4.  assuring  that  resources  are  obtained  and  used  effect Jvv:5y 
and  efficiently  in  the  accompl  i  shment  of  the  organ  f  2  a  : 'wr^*n' s 
objectives. 

(     )     29.    Management  report  is  one  kind  of  device    used  to  carry  out  t^e 
control  function.    What  should  such  reports  include? 

1.  brief  comparison  of  actual  progress  against  p.anned  progress; 
improved  situation  resulted  from  previous  cor  rect  i  ve  act  ?  Cjns' 
and  future  status  or  progress  of  the  project. 

2.  details  of  task  compl  et  i  on  whi  ch  resul  ts  in  the  present 
status  of  the  project  and  prediction  of  future  status. 

3,  detailed  explanation  of  correct ive  act  ions  recommended 

in  order  to  compl ete  all  the  tasks  wi th i n  the  t ime  frame. 

4,  resource  shortage  due  to  unexpected  and/or  uncontrol 1 ed 
c  i  rcumstances . 

(    )     30.    What  is  the  advantage  of  study i ng  a  models  such  as  the  '^genera  1  i zed 
project  management  model_,"  in  a  session  on  project  management? 

1.  presents  an  ideal  situation. 

2.  allows  the  student  some  freedom  to  speculate  upon  more 
useful  act ivi ties . 

3^     presents  a  summary  of  the  nature  of  tasks  involved. 
4.     forces  the  project  manager  to  fit  his  program  to  a  simpli- 
fied and  useable  framework. 
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Cot  lege  of  Eciucatioh 
Th e  Oh  t o  5 ta  te  Un \ ver  s  i  t y  * 
ColuiTibus^,  Oijlo  U32IO 

Project  Kanngement  fU^^ctica  Sc»i?le 


Directions:    On  each  page  of  this  booklet  you  vv] 1 ]   find  a  different  concept 
to  be  judged  and  beneath  it  a  se-C  of  sc-ilcs.    You  are  to  rate 
the  concept  on  c>acn  of  thesi:  sca?es  in  order. 


If  you  feel  that  the  concept  rjt  the  top  of  the  paga  is  very  close] y 
rel eted  to  one  end  of  the  scale,  place  your  ch^ck  mark  35  follows: 

fair     .  :^      ^    J....^..   .  -.^  unfair 


(f  you  feel  that  the  concept  is  q^jM  te  closeW  related  to  one  or 
tho  other  ehd  of  the  $c^!e  (but  not  extrerric)  you  should  ""pi  ace  your  check- 
mark ?is  follows: 

strong   x^_^:  ^  _  ^  weak 

If  the  concept  seems  only  ,,sj  i ght »: y  rol ated  to  one  side  as  opposed 
to\  the  other  (but  it  is  not  reaPly  neutral  )j  then  you  should  check  as 
follows: 

active  :  x  :  ^-.^^  ^^^^  ,  passive 

■  '  '  .  .  •  f. 

?f  the  concept  J nej^li^J^J..,  or  the  scale  is  j  rrel eyant  to  the 
concept,  then  place  your  check  mark  in  the  middle  space, 

IMPORTANT:     Place  your  check  marks  in  the  roiddh?  of  spaces ;  not  on -boundaries . 

Make  each  item  a  ^separate  and  independent  judgement.    Do  not 
■worry,  or  pu^.^le  over  individual  items,     it  is  your  first 
.  impression^  the  inwdiato  fee HngiS;  about  the  Items  that  we 
want. 
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changeable 


COKTROl. 


■     7^      .       .  .•;        ,4r«,i*,wi,-*^y    ^,iu„„0.,,.^u^    ^^^--^^w.^^    -n.««4«.r^«»**««    ***#«<*w*-«''u-*.'«>  .--i^w*-.*  ^w-^u»^'-f-- 

V. Stiff Icient  insufficient 
soothi ng  :  J ,_.„^L-_-^  aggravated  : 


Interpreted  '  ;  '  :  :  ■  r  :  r  :  unexp'iained 
useful  :         :         :         J         :         :  useless 

'  ...  .  .  _       _      -  ^      -^^-—^m^,t.^Mymmm  M*t(ffn  ^<H»X*it*i#MBiaai>» 


concise'  ^  •   '  .  .  :. ' -...J.....  '  v :^_.„ ■ 


1  htent  ional  :         :     '     >        '  .^^ ' :;.uni  ntent  ional 


complex:  . :    '  "  :  ^       '  :  V  '   .  •  r . '  ^.^  ^ ^  ^■  simple  ^ 

Stable-  -■■    ■  •     ■  i     ^■■''y.:'^''^     i  '^'^  ■■'■■^    r    ■  ^   :  :    ■  /^changeable 

'   ^    .       ..  ...         ....  ,.   .  ..     ■  *<M'-«Br»W»J-w»«iM.  «»»*««»i>^K»irf-*-. 


rationa)  r         r         ;         r         :         Y  V_  Iniuitive 
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:4 


benef tcial 
^'jf  f  Jci  ent 
soothing 
Interpreted 
useful 
authentic 
conci  se 
conslratned 
intentional 
complex  . 
stable 
rat  ional 


harmful 

Insufficient: . 

Licjgraysted.  ■ 

unexplai  ned 

useless 

r?>c  si  mi  le  ;  . 

diffuiie 

i!n  intentional 
simple 
changeable  . 
i  ntui  t ive 


ERIC 


SIHTAR  EXERCiSE 


r 

3 


beneficial 
suf  f  reierit 
&QOthi  ng 
Inter pre ted 


useful 


■  authent ic 
cone  I se 
cons t ra i ned 
Intent  Id hcjI 
compl ex 
stable 
r<3 1  t  onal 


■iMi*»      1l-n»iiM«rTP»  Hit  .M^Wiwr 
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*-*  r  *;«£r  i:>  I  f  ^  ^  <=f 
_  ^  «Ht  ^  f:>  1 


EKLC 


Elem.  Form  - 

THE  VAN  PiT  SERIES  -^™ 

FOR  GRADES  4  -  5  AND  6 

^  zur  name  h^re  .  -   -  — ^--.'^  ^ —   ■ — - — '- — ■■  

•ite  your  grade _^  ^  Date  :  — ^  Schooi.^   ■ 

DIRECTIONS 

Sometimes  when  we  read  something  we  th 
times  when  v/e  hear  sornething  we  think  about  that,  too;  ^  W 
ask  ourselves,  "Is  it  true?"  Often  we  say,  "That  isn't  true."  At 
other  times  v/e  can't  make  up  bur.  minds.  We  ore  uncertain. 

We  hove  some  stories  for  you  to  read.  After  each ^stor^ 
will  find  some  sentences.  We  would  iike  to  know  what  you  think^ 
about  them.  You  don't  hcve  to 

should  think  about  the  story  and  then  think  about  each  sen^ 
If  you  think  the  sentence  is  true,  draw  a  circle  around  the  "T"; 
if  you  think  the  sentence  is  f aise/  draw  d  ci rcte  around  T:he  "F";' 
and  if  you  can't  teli  whether  it  is  true  or  false,  draw  a  circle 
around  the 

If  you  caninot  read  it  or  understand  it/  raise  your  hand  and 
your  teacher  will  help  you  to  read  the  sentence.  Make  only  one 
(I)  circle  for  each  sentence.  Here  is  your  chance  to  show  how 
well  you  think. 


Copyright  oppiied  for  by 

THE  MODERN  EDi^ 

Box  26,  Bronxville,  N.  Y, 


THE  FIRST  STORY 

Dorothy  and  her  best  friend,  Sally,  went  for  a  hike;  in  the 
woods.  While  th^y  were  walking  along  they  saw  o  toad  on  the 
path.  Sally  stooped  to  pick  it  up.  'lOh/don't  touch  it/'  said 
Dorothy,  "My  brother  said  that  if  I'tduched  a:  tcoC'  I  would  get 
warts."  - 


Draw  a 

circle 

around  T,  if  you  think  the  sentence  is  true 

Drciv^  G 

circle 

around  F,  if  you  think  the  sentence  is  folse  • 

Draw  a 

circle 

around  ?,  if  you  can't  tell  if;  it  is  true  or  false 

T 

'  f 

? 

1 . 

Dorothy  liked  Sally. 

T 

'  i  .  . 

E 

■,  r 

^ , 

4  3  It:  oUf  1  W  CJb  bi  ( 1  ni  r  ly  lilt:  CJvjy  Ul    lite  ni                    .  ' 

T 

F 

? 

3. 

Dorothy's  brother  got  v/arts  because  he  picked  up  a  toad. 

T 

F 

? 

4. 

Sally  sow  the  toad  in  the  tree. 

T 

F 

? 

5. 

The  girls  took  the  toad  home. 

T 

F 

? 

-  6; 

Dorothy  warned  Sally  not  to  touch  the  toad. 

t 

F 

?  . 

They  saw  the ^tbajd  while  they  were  eating  their  lunch. 

T 

.  S.- 

Sally  will  have ^orts  on  her  hands  after  handling  the  toad. 

THE  SECOND  STORY 

•  . 

Sally  and  Dick  were  talking'  about  Jane,  who  was  in  their 
room  in  school.                                                                                 y  '  ■ 

''Don't  leave  your  lunch  money  Ground  when  Jane  is  there/'  * 
said  Dick.''."- "  '^r-:,- ^-  • 

"Why?"  asked  Sally.  *    .                  ^       ^     .  ^  . 

"Well/'  said  Dick/  "her  brother  is  in  my^  sisterV^ 
cQugHFhim^  s^^                                  !f  he  is  like:  that,  then 
there  rnust  be  something  wrong  with  the  whole  family/' 

Draw  a 

circle  around       if  you  think  the  sentence  is  true 

Draw  a 

circle 

around  F,  if  you  think  the  sentence\is  false 

prow  a 

circle 

around  ?/  if  you  can't  tell  if  it  is  true  or  false 

Sally,  Dick  and  Jane  ail  went  to  the  some  school. 

■ 

;  10. 

Jane  took  SoHy's  lunch  money. 

T 

■'F 

Jane's  brother  was  caught  taking  money. 

■12. 

Dick  WGrned  Sally  about  Jane. 

T 

13: 

Saliy  and  Dick  wisre  helping  Jane. 

T 

F 

:  14. 

Solly  always  bought  her  lunch.  : 

F 

■:';■■>■■■■■: 

15. 

J  one  wi  1 1  to  ke  th  i  ngs  too;  because  her  b  rother  takes  th  i  ngs. 

F 

16. 

Jane  did  not  have  very  much  money:  / 

ERIC 


THE  THIRD  STORY 

^  Harold  and  his  friend,  Tony,  Avere  standing  on  the  corner  of 
the  street.  Two  men  walked  by.  As  they  passed  one  man  said 
_Our  navy  lost  over  5000  men  in  this  last  battle  in  the  Pacific' 
They  say  that  v/e  won't  have  enough  boats  or  men  to  take  the 
iplace  of  those  we  have  lost." 
^"Gee,"  said  Harold,  '■'it  looks  pretty  bad  for  us." 
''Oh,  I  don't  know,"  said  Tony.  "Lost  week  !  heard  that  we 
had  sunk  more  than  sixty  enemy  ships." 


Draw  a  circle  around  T,  if  you  think  the  sentence  is  true 
Draw  a  circle  around'  F,  if  you  think  the  sentence  is  false 
Drew  a  circle/  arouhd  ?,  if  you  can't  tell  if  it  is  true  or  false 


?  17.  The  men  who  passed  y/ere  talking  very  loudly. 

^  ^8.  The  navy  lost  the  battle  the  men  were  talking  about. 

^  ^9.  The  boys  v/ere  frightened  by  what  the  men  said. 

7  ^0.  It  is  possible  that  the  two  men  were  spies. 

T  21.  The  United  States  navy  sank  more  ships  thon  the  enemy. 

^  22.  This  battle  in  the  Pacific  Ocean  was  the  lost  battle  in  the 

■  .■  . ;  ■'.  war.'  ■ 


?  ;  .  23 

^  24.    The  two  boys  said  just  about  the  same  thing. 


The  boys  were  waiting  for  the  bus  when  they  heard  the 
conversation. 


THE  FOURTH  STORY 

At  Scott  School  there  is  a  fine  ball  team.  This  school  won  the 
championship  lost  year.  Steve  goes  to  Scott  and  his  frien^  Jack 
goes  to  Jefferson  which  does  not  have  a  very  good  team. 

"Scott  is  a  better  school  than  Jefferson,"  said  Steve  one  day. 

'■I  don't  think  so,''  replied  Jack.  "I  like  our  SGhool:'' 

"Ours  is  the  best  because  we  won  the  championship,  didn't 
we?  Your  team  is  terrible,"  answered  Steve. 


Draw  a  circle  around  T,  if  you  think  the  sentence  is  true 
Draw  a  circle  around  F,  if  you  think  the  sentence  is  false 
Draw  a  circle  around  ?,  if  you  can't  tell  if  it  is  true  or. false 


?  25.  Scott  won  every  gome  they  played  last  year. 

?  26.  You  would  learn  to  be  a  good  sport  if  you  went  to  Scott. 

?:  27.  Scott  will  beat  Jefferson  this  year. 

?  28.  Jock  thinks  his  school  is  a  good  school. 

?    ;  29.  The  Jefferson  teann  is  terrible. 

?  30.  Scott  is  the  finest  school  in  town. 

?  31.  Scott  will  beat  Jefferson  because  they  have  a  finer  school. 

?  '    32.  Jack  would  like  to  have  Jefferson  win  the  championship. 


THS  F8FTH  STORY 

.  While  walking  to  school  one  day,  Fred  said  to  Jim,  "!  don't 
thmk  that  those  fellows  ot  the.  Johnson  foctory  hove  a  rioht 
to  strike."         :  ' 

"But  my  fo-'ther  works  there  ond  he  says  that  they  have.  They 
don  t  pay  them,  enough,"  answered  Jim. 

^'You're  ail  wrong,"  said  Fred,  "i  read  in  the  newspaper  about 
the  whole  tning.,^ond  it  said  that  they  were  paid  enough." 


Drow  a  circle  around  T,  if  you  think  the  sentence  is  true 
Draw  o  circle  around  F,  if  you  think  the  sentence  is  false 
Draw  a  circle  around  ?,  if  you  can't  tei!  if  it  is  true  or  false 


T 

F 

33. 

T 

F 

? 

■  "'34. 

T 

F 

? 

35. 

T 

F 

•  ?  - 

36. 

T 

F 

■> 

37. 

T 

F 

? 

38. 

T 

F 

? 

39. 

T 

F 

? 

40. 

The  boys  did  not  agree  about  the  strike. 

You  should  believe  Jim's  father  before  you  believe  the 
newspaper. 

Listening  to  both  boys,  you  can  be  sure  that  each  one  was 

doing  his  own  thinking.  ■ 
It  is  never  right  for  men  to  strike  during  wartime. 
Jim's  father  wanted  the  men  to  win  the  strike. 
If  it  was  in  the  nev^spaper,  then  it  must  be  true. 
Fred's  father  does  not  Iwork  jn  the  factory. 
Jim  doesn't  believe  v/hat  he  reads  in  the  newspapers. 


THE  SJXTH  STORY 

^There  ore  twenty  children  in  thejfourth  grade.  The  children  in 
the  tourth  grade  drink  milk  of  school  every  day  at  noori.  There 
are  four  glasses  of  milk  in  each  quart.  This  tells  how  many  quarts 
of  milk  tnis  class  dronk  every  day  lost  week 
MONDAY  TUESDAY  WEDNESDAY  THURSDAY  FRIDAY 
-  ^9"^°^^^..  -   .1^  Quorts  ,  _3  Quarts.,     6  Csiprts 


Draw  a  circle  around  T,  if  you  think  the  sentence  is  true  '"^ 
Draw  Q  circle  around  F/ if  you  think  the  sentence  is  foise 
Draw  a  circle  a  round  ?/  if  you  can't  tell  if  it  is  true  or  false 


T 

F 

45. 

T 

F 

42. 

it*.'.' 

• 

F 

'43. 

T 

F 

44. 

T 

F 

45. 

T 

F 

46. 

T 

F 

47. 

T 

F 

? 

48. 
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Each  student  in  the  class  drank  two  glasses  of  milk  on 
Wednesday, 

Eve  ry  c  h  i  j  d  d  ra  n  k  twi  ce  as  m  uc  h  m  i  i  k  i  n  c  I  ass  on  Wednes- 
day as  on  Monday.  ; 

Gn  Thursday  some  of  the  children  in  the  class  did  not  want 

■  any,milk.;, 

There  was  enough  milk  for  every  child  in  the  class  to  have 
a  glassful  on  Tuesday. 

On  Thursday  and  Friday  the  children  drank  more  milk  in 
:  school  than  they  did  on  Monday  and  Tuesday. 
Four  children  did  not  get  any  milk  on  Tuesday.  ; 
The  child  ren  d  i  d  no  t  wont  much  mi  Ik  on  Thursday  because 
they  had  hod  a  lot  on  Wednesday. 

Some^children  hod  two  full  glasses  of  milk  in  school  on 
Friday. 


CRITICAL  THII.KING  TEST 
Ethel  Wildey  Haw 

S;-r7T£7?-  ^'f '  '°  "^1^  judge  statements  based  upon 

facts  which  are  preseritGd  to  you.       m  some  cases,  you  can  tell  from  the  f,,c^s 
Sivcn  that  a  statement  is  false  or  true.      In  other  cases,  you  can  ^av  that 

:Jtl1n''  '^t^e'^'^^r-n  --^^  facts  sug  eL  this  b^J  do  no  Lue 

..It  ccrtdin.      There  are  still  other  cases  where  you  are  not  piven  enouph  fnr»-..  ^^ 
.alee  It  possible  for  you  to  say  anything  about  .L  ,ruth  or  Lnitrrf'the  ^trtelnc. 

If  you  can  tell  from  the  facts  given  that  a  statement  is  Itrue,  mark  T  with  an  X. 

if  you  can  tell  from  the  facts  given  that  a  statement  is  fal_se.  mark  F  with  an  X. 

JLZ^  ^Ifrfp^^J^h  :"r^"^^^  ^^^r^-^  ^°  -^^^        certain,  i.  is 

-  false  but  do  not  make  it  certain,   i.  is  ; 

nK^Vv!""^  are  not  enough  facts  given  to  make  it  possible  for  you  to  say  anvthin- 
about  the  truth  or  falsity  of  the  statement,  mark  NE  with  an  X  ^"ythmg 

ti^lJ^^"'^)^  '^'  ^^  and  F  will  be  shown  at  the  top  of  each  page  of 

SoStS'     ^^brother  and  sister  raised  tulips  to  enter  in  the  annual  flower  shot, 
care  ^^Tl^  '^'T  some  in  clay.      The  .tulips  were  tended  with 

io:::on  tie  pr?^r        ''"^  for  tHe  flower  show.      The  tulips  which  had  grown  in 

PT    NE    PF  ^F^     1.    The  flower  show  was  held  once  each  year. 
--Urue.    -The  paLayraph  tells  xhac  it  was  ah  annuar  floxjer  show.)  '  "'^ 

T  ME    PF    F      2.     Plants  grow  well  in  loam. 

(Probably^true.    The  paragraph  tells  that  tulips  grew  well  in  loam.     It  does  not 

nrnh.Mrf  I  ^  u"^' \  Since  tulips  are  plants,  what  is  true  of  tulips  is 

probably  true  of  other  plants  as  well.) 

FT.  NE    PF   X     3.  .  All  of;  the  bulbs  were  planted  in  loam 
(False.     The  paragraph  tells  that  some  bulbs  were  planted  in'clay.) 

'^nrJ^   ^  ^      ^'    ^^liPS  grow  better  in  loam  than  in  .any  other  kind  of  soil 

(ilot  enough  facts  given.     The  paragraph  tells  about  only  two  kinds  of  soil,  loam 
and  clay.  .    It  does  not  tell  about  other  kinds  of  soil .) 

,T    PT    NE^  ^  F      5.     If  the  brother  and  sister  raise  tulips  next  year,  they  will 

not  plant  any  in  loam. 
(Probably  false        The  paragraph  tells  that  the  tulips  which  grew  in  loam  won  the 
prize.      We  would  judge  from  this  that  they  would  probably  try  loam  again. 


DO  NOT  TURN  THIS  PAGE  UNTIL  YOU  ARE  TOLD  TO  DO  SO 


-  2  - 
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T      True  according  t:o  Ll-ie  facts  givon, 
PT  .  PLobRbly  Ci-ue  from  Lhe  facts  given . 
NE    Not  enough  facts  given. 
Pr.    Probably  [alec  from  the  facCc  given. 
F    p False  according  to  the  facts  given." 


PROBLEM  I 


.n,.l/''\'^'''''^^^-^°"''  '^^f"'"'''  °^  ^""'"^  "'""''^         iiSi.tning  is  very, very  smnll,  thi.> 
small  .chance  is  something  to  think  about.      If  you  are  out  in"   he  open  vmin  a 

■  TeT  tlcTsT-^^'^^^^  you  Should  stay  away  fro.  trees,  high  places:'and  ;:    e  fen- 
ces. t^ocausG  .he..e  are  m  greatest  danger  of  bei:,;-  struck  by  lightning        If  there 
IS  a  building  near,  go  inside  to  keep  dry.  "  >   xigncning.       ii  thcie 

T    PT    NE     PF    F      1.  YOU  will  not  be  struck  by  lightning  if  you  stay  inside  a 

building  during  thunder  r'orms. 

P  ?T    KE    PF    F        2.  A  tree  on  top  of  a  hill        more  likely  to  be  struck  by 

lightning  than  a  tree  in, a  valley. 

T    PT    NE     PF     F      3,  Within  your  whole  lifetime  you  will  not  be  struck  by 

lightning.  .  : 

T    PT    NE    PF    F      4.  If  there  is  no  building  near  when  a  thunder  storm  begins. 

you  should  go  under  a  tree  to  keep  dry, 

T    PT    WE    PF    F      5.  A  farner  would  not  leave  a  cow  tied  to  a  wire  fence  durin- 

a  thunder  storm. 

T    PT    NE    PF    F      5.  Lightning  never  strikes  twice  in  the  same  place. 

T    PT    NE    PF-  F  ;    7.  Lightning  will  strike  whom  it  will,  and  there  is  nothing 

that  can  be  done  about  it.  '  " 


Ao..,-^°"°T  'J  °^  '^'^  world  learned  how  to  use  rubber,  the 

Cou'^bn^  Indians  were  making  things  out  of  it.      On  one  of  his  trips  to  America. 
Columbus  saw  Indians  play:=.ng  games  with  rubber  balls.      The  Indians  also  made 
water  jars  and  shoes  from  rubber, 

8.  We  do  not  know  v7hen  the  Indians  first  began  to  use  rubber 

9,  The  Ind  ans  made  no  important  discoveries. 


T 

PT 

NE 

PF 

F 

T 

.  PT 

NE 

PF 

F 

T 

■PT 

NE 

PF 

F 

T 

PT 

NE- 

PF 

F 

T 

PT 

NE 

PF 

F 

were  playing  with. 
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T      True        PT      Probably  true    .      KE      Not  enough  facts  given 

PF      Probably  false  F  False 

phoblem  III 

A  number  of  years  ago,  it  was  discovered  that  bread  made  from  white  flour 
did  not  contain  the  vitamins,  minerals,  and  proteins  that  were  in  whole  xjhcat 
bread.      Many  people  still  used  white  bread,  because  they  had  become  used  to  the 
taste  of  it.      However,   some  of  the  minerals  and  vitamins  are  added  to  xjhite  flour 
so  people  can  have-x-jhite  bread  if  they  want  it  and  still  get  many  of  the  same  food 
values  they  X70uld  get  from  whole  wheat  bread . 


T 

PT 

WE 

PF 

F 

13. 

Old  habits  are  not  easy  to  break. 

T 

VT 

NE 

PF 

F 

;,  14. 

White  flour  nov;  contains  all  of  the  food  values  of  whole 
v;heat  flour .  ■ 

T 

PT 

NE 

PF 

F 

15. 

Whole  wheat  bread  is  more  healthful  than  white  bread. 

T 

PT 

NE 

PF 

F 

16. 

Science  can  solve  all  of  our  food  problems. 

T 

PT 

NE 

PF 

F 

17. 

Now  that  we  have  enriched  V7hite  bread,  fewer  loaves  of 
vjhole  xaheat  bread  are  sold.  - 

T 

PT 

NE 

PF 

F 

18. 

People  who  like  white  bread  are  not  pleased  to  have 

vitamins  and  minerals  added  to  it, 

PROBLEM  IV 

Since  the  last  ice  age  there  have  been  cycles  of  V7armer  and  colder  climates* 
The  cycles  are  thought  to  be  caused  by  changes  in  the  amount  of  carbon  dioxide 
icuthe  :air.       Scientists  are  trying  tc^^^^^^  what  climate  was  like  at  different 

times  in  the  past  by  studying  the  air  which  is  trapped  in  the  ice  of  glaciers,  . 
They  are  studying  the  air  in  the  huge  ice  sheets  of  Greenland  v/here  some  of  the 
ice' may  be  20,000  years  old  or  more.      They  have  found  that  the  amount  of  carbon 
dioxide  in  the  air  of  some  layers  is  twice  as  much  as  in  present-day  air. 

T    PT    NE    PF    F      19.     The  climate  of  a  region  does  not  alx^rays  stay  the  same. 

T    PT    NE    PF    F      20.     Scientists  will  learn  what  the  climate  of  Greenland  x^as 

like  5,000  years  ago. 

T    PT    NE    PF    F      21.     When  there  is  more  carbon  dioxide  in  the  air,  the  climate 

is  colder . 

T    PT    NE    PF    F      22.     The  scientists  fear  that  the  ice  sheets  of  Greenland  will 

be  gone  before  they  can  finish  their  study. 

T    PT    NE    PF    F      23.     Some  of  the  ice  now  in  Greenland  was  there  when  Columbus 

discovered  America, 

T-  PT    NE    PF    F      24,    At  one  time  Greenland  was  as  warm  as  Florida  is  now, 


EKLC 


T  ,  True 
PT      Probably  true 
NE      Not  enough  facLs  given 
PF      Probably  false 

F      False  ' 


PROBLEM  V 


People  studyins  human  growth  found  that  a  large  group  of  American-born 
Japanese  children  in  California  grev7  at  the  same  rate  as  a  large  group  of  other 
American  children  living  in  a  x^ealthy  neighborhood  in  Ohio.      They  alsofound  ' 
that  the  American-born  Japanese  children  grew  at  a  faster  rnte  than  a  lar-e  group 
of  Japxinese  children  living  in  Japan,      Uhen  the  diets  of  the  three  groups  of 
children  were  studied,  it  was  found  that  the  children  living  in  America  had  more ■ 
and  better  food  than  the  children  living  in  Japan. 

T    PT    NE    PF    F    25.  Good  food  is  important  to  growth. 

T    PT    NE    PF    F    26.  The  Japanese  children  in  Japan  grew  more  slowly  than  the 

American-born  Japanese  children, 

T    PT    NE    PF    F    27.  The  climate  of  California  is  more  healthful  than  the  climate 

of  Ohio . 

T    PT    NE    PF    F    28,  By  nature,  Japanese  people  grow  more  slowly  than  people  of 

other  races.  ..  . 

T    PT    NE    PF    F    29.  If  the  children  living  in  Japan  had  been  born  and  reared 

in  America,  they  would  have  grown  as  fast  as  the  Americaa- 
born  children. 

T    PT    NE    PF  :  F    30.  The  wealthy  Ohio  children  had  been  spoiled  by  soft  living. 

.  T    PT    NE    PF    F    31.  Some  of  the  differences  between  people  of  different  countri^^s 

are  caused  by  differences  in  diets. 

PROBLEM  VI 

Susan  Hall  wanted  to  do  well  on  the  spelling  test  she  would  have  on  Friday. 
She  had  not  done  well  on  her  spelling  tests  in  the  past.      Her  mother  said. 
'Why  not  do  some  studying  each  evening  instead  of  waiting  to  do  all  of  your  study- 
ing on  Thursday  evening?^'      Each  evening  that  week,  Susan  studied  spelling. 
On  Friday  she  made  a  perfect  score  on  the  spelling  test.' 

More  can  be  learned  by  studying  at  home  than  by  studying  at 
school . 

From  then  on  Susan  tried  to  do  some  studying  each  day 
instead  of  vjaiting  until  just  before  the  test... 
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32, 
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33. 
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34. 
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PT 
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PF 
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35. 

T 

PT 

NE 
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F 
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ERIC 

PT 

NE 

PF 

F 

37. 

T  True 

PT  Probably  true 

NE  Not  enough  facts  given 

PF  Probably  false 

F  False 


PROBLEM  VII 


Cement  has  been  used  for  building  for ,  thousands  of  years .       Romans  of  }ony^ 
ago  used  it  for  making  roads  and  walls  of  buildings.      Many  of  the  walls  are  " 
still  standing  and  some  of  the  roads  made  by  the  Romans  2000  years  ago  are 
still  used  today. 

Wear,  weather,  and  changing  temperatures  are  hard  on  cement,  yet  well-made 
cement  can  last  for  ages. 

T.  PT    NE    PF:    F    38.  Some  of  the  old  Roman  roads  are  still  in  use. 

T    PT.  NE    PF    F    39.  If  the  winters  of  Rome  ^^7ere  more  severe,   the  cement  roads 

would  not  have  lasted  so  well. 


T 

PT 

NE 

PF 

F 

^0. 

The  oldest  things  are  the  best. 

T 

PT 

NE 

PF 

F 

41. 

Some  important  Roman  roads  were  not  used  for  centuries. 

T 

PT 

NE 

PF 

F 

42. 

The- Romans  have  never  been  thought  to  be  good  builders. 

T 

PT 

NE 

PF 

F 

43. 

Cement  was  discovered  by  an  American  scientist. 

T 

PT 

NE 

PF 

F 

44, 

Some  of  the  Roman  roads  have  been  in  use  for  more  than' 

2000  years , 
PROBLEM  VIII 

"    Farmers  have  found  an ineKpensive  way  of  getting  rid  of  some  of  the  cater- 
pillars that  eat  their  crops.      There  are  small  flying  insects  which  lay  their 
eggs  inside  the  bodies  of  the  caterpillars .       If  early  in  the  season  the  farms 
allow  tho  caterpillars  to  live  on  their  crops,   the  young  insects  living  inside 
the  caterpillars  will  soon  grow  up  and  cut  their  way  out  of  the  caterpillars' 
bodies.      In  doing  this  they  kill  the  caterpillars;      The  new  flying  insects 
will  then  lay  their  eggs  in  other  caterpillars .       In  this  x^ay;  many  cater- 
pillars are  killed  at  little  cost  to  the  farmers. 

T    PT    NE    PF    F    45.  It  would  be  cheaper  for  the  farmers  to  poison\>the  cater- 
pillars than  to  let  them  eat  the  crops  even  for  a  short 

'■■■time,;-/  ' "  '  . 

T    FT    NE    PF    F    46,  Insects  are  enemies  of  mankind. 

T    PT    1^    PF    F    47.  If  the  farmers  >ill  the'  early  caterpillars,  there  V7ill 

he  fewer ■ flying  insects  later  in  the  season.:  . 

T    PT    NE    PF    F    48.  When  people  destroy  living  things ,  they  upset  the  balance 

of  nature . 


EKLC 


GO  RIGHT  ON 


-6- 


True 

Probably  true  ^ 
Not  enough  facts  given 
Probably  false 

False  . 

For  inany  hundreds;  of  years  people  made  paper  by  hand  from  cotton    and  linen 
rags.      Then  in  1789,  a  Frenchman  named  Louis  Robert  invented  a  paper  machine,  : 
Wherever  these  machines  xvere  set  up,  rags  were  quickly  turned  into  paper  and 
soon  there  was  a  shortage  of  rags. 

Frederic  Keller,  a  German,  remembered  that  a  French  scientist  had  suggested 
making  paper  from  vrood  as  wasps  do.      After  watching  wasps  make  paper  from  bits 
of  wood,  he  and  a  friend  tried  it  and  finally  found  a  vjay  to  make  paper  from 
wood  pulp.      This  solved  the  rag  shortage,  but  soon  there  Xv^as  another  shortage  - 
a  shortage  of  trees, 

Keller  was  not  successful  the  first  time  he  tried  to. make 
paper  from  wood. 

The  French  .are  better  scientists  than  the  Germans. 

Many  trees  were  cut  down  to  supply  wood  pulp--f or  making 
paper , 

Most  of  the  paper  which  is  used  today  is  made  from  rags. 

The  shortage  of  rags  for  paper  was  worse  in  Germany  than 
in  other  countries  of  Europe. 

People  can  learn  from  studying  other  creatures . 
The  Germans  are  good  observers  of  nature. 


There  are  many  kinds  of  storms.       Some  storms  are  destructive ,  but  most 
storm.s  are  very  useful.      They  help  to  circulate  the  air,      They  bring  rain. 


They 

clear 

the 

air  of 

dirt,   soot,  and  smoke. 

T  PT 

NE 

PF 

F 

56 . 

Some  storms  do  great  damage. 

T  PT 

NE 

PF 

57.. 

There  is  some  good  in  every  evil. 

T  PT 

NE 

PF 

F 

58. 

Storms  are  more  often  destructive  than  useful. 

T  PT 

NE 

PF 

F 

59, 

The  next  storm  to  pass  through  your  neighborhood 
will  be  a  destructive  one. 

T  PT 

NE 

F 

60. 

If  it  weren't  for  storms,  this  world  would  not  be  such 
a  pleasant  place  to  live. 

T 

PT 
NE 
PF 
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T  PT  NE 

T  PT  NE 

T  PT  NE 

T  PT  NE 

T  PT  NE 

T  PT  NE 

T  PT  NE 
PROBLEM  X 


PF  F  49. 

PF  F  50. 

PF  F  51. 

PF  F  52. 

PF  F  53, 

PF  F  54. 

PF  F  55, 


ERIC 


T  True 

PT  probably  true 

NE  Not  enough  facts  given 

PF  Probably  false 

F  False 


PROBLEM  XI 


Tlicre  is  more  carbon  in  hard  coal  than  in  soft  ccal.      Hard  coal  burns 
with  almost  no  smoke  and  leaves  very  few  ashes.     Soft  coal  rankes  a  lot  of 
smoke  and  soot  when  it  burns.      Many  large  factories  burn  soft  coal  becaus^c 
it  is  cheap  and  plentiful, 

T    PT    NE    PF    F    61.  People  like  to  live  near  factories  that  burn  soft  coal. 

T-    PT    KE    PF    F    52,  The  more  carbon  there  is  in  coal,  the  more  smoke  it  makes 

when  it  burns. 

T    PT    NE    PF    F    63,  Factory  ouners  care  nothing  about  the  health  of  the  people 
■  who  live  near  the  factories.  - 

T    PT    NE    PF    F    64,  Factory  owners  would  like  to  use  hard  coal. 

T    PT    KE    PF    F    65,  Soft  coal  is  cheaper  than  hard  coal. 

PROBLEM  XII 

The  corn  plant  bears  seeds  in  the  middle  of  tHir  stalk,  while  other  cereals 
bear  seeds  at  the  tip.      Since  the  middle  of  the  stalk  is  thicker  and  stronger 
than  the  tip,  it  can  hold  the  weight  of  a  larger  ear.      From  this  we  would  tliiink 
that  corn  might  produce  more  than  other  cereals.      It  is  well  known  that  record 
yields  of  wheat  are  not  likely  to  be  more  than  100  bushels  per  acre,  v;hile  { 
record  yields  of  corn  are  more  than  300  bushes  per  acre,  .  *  . 

The  earliest  corn  plants  bore  their  seeds  at  the  tip  of  the  stalk.  When 
the  place  of  bearing  the  seeds  was  changed,  the  corn  plant  became  more  useful 
to  people.      At  the  same  time,  the  corn  plant  became""  less  able  to  scatter  the 
seeds.      It  is  therefore  less  able  to  live  in  a  wild  state. 

T    PT    NE    PF    F    66.  All  cereals  bear  their  seeds  at  the  tip  of  their  stalk. 

T    PT    NE    PF    F    57,  Farmers  who  raise  corn  make  about  three  times  as  much 

profit  asFarmers  who  raise  wheat. 

T    PT    NE    PF    F    68,  Next  year  the  record  yield  of  wheat  in  bushels  per  acre 

will  be  greater  than  the  record  yield  of  corn.        /  . 

T    PT    NE    PF    F    69.  If  corn  were  no  longer  raised  by  people,  it  would  grow 

well  in  a  wild  state.  ■  ""■      ■  ; 

T    PT    NE;   PF    F    70,  Corn  plants  now  look  very  different  from  the  earliest  corn 

plants, 

T:  PT    NE    PF    F    71 .  Corn  plants  now  are  more  productive  than  earlier:  corn.: 

GO  RIGHT  ON  '^"^ 


T  l:'ue 

PT  Probably  true 

NE  Not  enough  facts  given 

PF  Probably  false 

F  False 


PROBLEM  XIII 

Hobbies  somGtimes  lead  to  discoveries.       For  example,  Lccviwenhook  vjho  lived 
in  Holland  about  300  years  ago,  had  the  hobby  of  grinding  and  polir^hing  pieces 
of  glass  into  mcgnlfying  lenses.       He  v;orked  so  carefully  that  his  lenses  were 
stronger  than  any  that  had  been  made  before.      He  looked  at  everything  he  could  . 
get  hold  of  with  his  new  lenses.      One  day  he  placed  a  drop  of  pond  war^r  under 
ills  lenses.      "      his  grea t  surprise:,  he  found  that  it  was  filled  v/ith  tiny  plants 
and  animals  e  had  discovered  a  nev;  world  of  living  things  that  had  never 

been  seen  be j  v  ie  , 


T 

PT 

NE 

PF 

F 

72. 

Leeuv/enhook  was  the  first  person  to  see  the  tiny  living 
things  in  pond  water. 

T 

PT 

NE 

PF 

F 

73. 

Every  person  should  have  at  least  one  hobby. 

T 

PT 

NE 

PF 

F 

74. 

Leeuwenhook  looked  at  a  drop  of;  blood  with  his  new  lenses. 

T. 

■  FT 

NE 

PF 

F 

75. 

Leeuwenhook's  lenses  uere  almost  as  good  as  those  he  could 
have  bought  in  the  finest  shops. 
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PT 

NE 

PF 

F 

76. 

Important  discoveries  are  most  often  made  by  chance. 
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PF 
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77. 

Soon  after  his  discovery,  Leeuwenhook  loot  interest  in 
his  1 ensGS . 

T 

PT 

NE 

PF 

F 

78. 

Leeuwenhook  -nade  the  first  magnifying  lenses. 

PROBLEM  XIV  • 

Ear-  year  there  are  many  more  people  living  on  this  earth  than  there  x^;ere  - 
the  year  before.      This  has  led  to  the  fer^.r  that  there  will  soon  not  be  enough 
food  to  go  around.      In  some  countries,  there  is  a  shortage  of  food  at  the  present 
time.    :  Foods  that  supply  proteins  and  vitamins  are  the  ones  fhat  are  most  likely 
to  be  scarce. >     Since  these  foods  are  very  important  to  the  health  of  children, 
several  groups  in  the  United  Nations  have  been  try ing  to  find  vays  to  supply 
them' cheaply:  in  places  x^^here  they  are  needed.      One  of  the  most  important  find- 
ings made  by  these  groups  is  that  vegetable  proteins  can  be  used  in  place  of  - 
animal  proteins  in  children's  diets. 

T    PT    NE    PF    F    79 .  Vegetable  proteins  cost  less  than  animal  proteins 
■T    PT    NE    PF    F    80,  Proteins  and  vitamins  are  not  important  to  grownups, 

T  PT    NE  PF  F  SI .The  population  of  the  world  is  increasing  rapidly . 

-T  PT.  .NE  PF  F  82,  Animal  prot.eins  are  not  necessary  to  the  health  of  children. 

^^T  PT    NE.  PF  F  83.  "We  should  take  care  of  present  needs  and  not  trouble  . 
■  ■  ourselves  ;about  the  future  ;    ■    .  .   ;  v;   .:  .  ■ 

T    PT    NE    PF    F    S^\.  There  is  no  .reason  to  believe  that  there  will  ever  be  a 

shortage  of  food  on' this  earth. 
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10. 

The  imp lem<ifnca Clou  of  a  strategy 

3  '  ;■ 
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Roles  of  the  e-saluatox 
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CI  RCE  Attitude  Scale  No.  1.4  N  .ir;u' 

Attitudes  toward  Educotionol  Evaluation.    Hclow  .uc  ;i  iKiriilu::  .jio;!'         v'.iluatiori  nj  c 'iii:  ca  rii^i:.i  1  proi-r.i-s  s  , 

A  proi^ram  c;iij  he  a  itrsson.  n  course,  a  ^\'ho\t  curriculuiii.  or  a:iV'  traiiMrii^  activ-t'.     <  ..n^jdc:  ca.-lt  stafe::st'iU  a>  a  scarniic;ii 
i)(  opiMifvji.    If         Ji^rc«i-  ..r  least  a  Inilr  l^ir  ua^li  the  ■■[atcmcra".  circle  ::;c:  ic^rcr  A.  .If  \ou  :iisa^i:rL;e  nvcn  a  ^rilc  f^:r  'A-:th 
the  sfatcjnciu.  circle  iht:  letter  D.    li  vun  ':.)ih  .iKTCt:  and  .lisa;:rcc.       u         uaw       npinu.n.  icavt-  trie  lertcTS  ufui:.,;  : c-.l. 

A     AGREE  D     DISAGREE  Blank  Neither 

■   L     A     D     "Jht-  major  pur{X)se  or  an  rJucationa!  eviiiuation  study  shiuilt!  he  to  ^.iihcr  lii tormati<„>n  liiat  be  iicln  J-t.« 

rhe  c(fuc:uins.,  '  ■ 

A     D     li  is  iin]M)riani  for  chf  proi^rarn  evaiuator  to  find  out  fiovv  well  varicuis  r>euplc  like  the  prot;rain. 

3-     A     D    (ienernily  speakinc.  :in  educational  program  should  be  evaluated  with  reference  to  one  or  inoT<:  "contror*' 
pro  t:  rams,  - 

i-     A     1.)     'rhe  evaiuator  ^ihou  ld^^ccept  :he  fesp'on  si  hi  I  i  ty  o  f  findi  n^t^  the  stron^L'Cst.  mosc'defcnsible,  arKTpuhliclv  attrac* 
live  poinrs  of  the  program. 


A  1.) 


in  evaluaunt^  a  program,  it  is  at  least  as  important  to  studv  and  report  on  the  r\;^es  oi  ceacf'inti  as  it  is 
study  and  re{:>ort  on  the  amount  of  learning. 


6.  :  A     n    The  evaiuator  should  drau'  a  conclusion  as  to  whether  or  not  the  goals  of  the  prL.vr:'m  are  worthwhile. 

It  is  more  important  to  evaluate  a  program  in  comparison  to  what  other  programs  :io  :^an  to  evaluate  it  with 
reference  to  what  its  objectives  say  it  should  do. 

D     Principals  and  superintendents  should  not  gather  data  about  the  quality  of  instruction  ip  the  classroom. 

The  task  of  putting  educational  objectives  into  writing  , is  more  die  responsibility'  of  the  evaiuator  duan  that 
of  the  educator. 

D    It  is  essential  that  the  full  array  of  educational  objectives  be  stated  before  the  program  begins. 

Evaluation  studies  would  improve  if  they  gathered  more  kinds  of  information,  even  if  at  the  expense  of 
gathering  less  reliable  informadon. 
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23. 

.A 

D 

I.  valuation  snouia  aid  an  educator  in  revising  his  goals  even  while  tne  prograna  is  in  progress. 

The  process  of  decision-maN'ing  about  the  curriculum  is  oi.';'  of  the  weakest  links  in  the  pre  sen 
the  schools 


his  competence  or. change  his  attitudes. 


.  Please  turn  over 


2.j..     A     I)     An  cvahiator  should  Htui  out  if  the  toac'liin^;  is  in  fact  die  kind  ih,ii  the  school  faculty  cxpcct.s  ii  to  be. 

25.  .A     I.)    Whether  or  not  nn  evaluation  report  is. any  tzood  sho'jld  be  decided  pretty  ;nuch  (ui  the  same  grounds  that 

research  journal  editors  use  to  decide  wht-tiier  ur  not  a  manuscripr.yshould  be  published. 

26.  A     n    The  main  purpose  of  evaluation  is  to  gain  understandiii  ;r  of  che  causes  ot  ^ood  instruction, 

27.  A     I")     Dcscriptiofi  and  value  judgment  are  f;'qual]y  important  componcius  of  evaluation. 

In  conductinL'  an  evaluation,  rhere  is  no  jusci ficatioa  for  the  exercise  of  subjective  jud^menrof  i\n\  kind 
bv  the  e valuator,  • 


for  evaluation  data. 

The  educational  e valuator  should  attempt  to  conceai  all  of  his  personal  judgment  of 
program  he  is  evaluating. 

The  sponsor  of  an  evaluation  .should  have  the  final  say-so  in  choosing  or  eliminating 
studied.  . 

The  main  purpose  of  educational  evaluation  is  to  find  out  what  methods  of  instructio 
learning  situations. 

34.   ,  A     D     Parents'  attitudes  should  be  measured  as  part  of  the  evaluation  of  school  programs. 

An  evaluator  finds  it  almost  impossible  to  do  his  jobu'ithout  intruding  upon  the  oper 
at  least  a  little. 
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D 

■  48. 

A. 

D 

in,structionai  research  study  would. 


42.     A    D.  The  leader  of  an  evaluation  team  should  be  a  teacher. 

The  entire  school  day  and  the  entire  school  experienci 
of  stated  educational  goals. 

An .  evaluation  of  an  educational  program  should:  inciud 
the  program. 

Every  teacher  should  h,  ."c  formal  ways  of  gathering  in 
of  his  ins  true  t.io^a^  program.     -  ■  ■ 


administrators,  teachers,  and  students  themselves. 


:49y  ?A.  ■  "  When  the.  evaliiaror  has  to  choose  between  helping  this  staff  run  its  pirogram  better  and  helping  educators 
'     -  .    "     ,        everywhere-undersiand  all  programs  a  little  better  he  should  choose  the  l}?tter.    v  ; 


INSTITUTE  PARTICIPANT  OPINIONNAIRE 


CIRCE 
SuTnmer  1970 


5-DIGIT  CODE 

DIRECTIONS:     The  following  items  have  been  written  to  enable  you  to  say  how  you 
feel  about  several  aspects  of  an  instructional  week  of  your  Institute.     Check  one 
response  category  for  each  question  and  conunent  if  you  wish  to  clarify  your 
response. 

Select  any  5-digit  code  number  and  use  it  throughout  the  Institute  on  these  opin- 
ioimaires. 

Yes      Yes-'*but*'        No    I  don *t  know 

1.     Are  you  enjoying  yourself  at  this               |     |          \     \          |     I  J  I 

Institute?  L — !  UJ  !  1  I — 4 

C^^■::•^^ient; -. 


2,     Are  you  getting  the  chance  to  talk 
to  the  staff  as  much  as  you  would 
like  to? 


In  your  opinion,  is  the  entire 
Institute  well-organized?  / 


Comment: 


□ n  n  n 
LJ     LJ  LJ 


Comment : 


4.    Has  the  general  administration  of 
this  past  week  been  v?eIl-organized? 


□  □  □  □ 


Comment: 


-2- 


Yes      Yes-'W*        No    I  don't  know 
5»    Have  you  been  getting  aLong  well  with       [     ]  j     1  [     |  j  | 

other  -narttrinarktci    at    fhp    Tnc^^^l♦^^i7  L— J  '       '  I— J  I  ■  ■  ' 


Other  participants  at  the  Ins titute? 

Coiranent: 


6.    Did  you  enjoy  the  instructional  staff           1          |     |           f    I  I  H 

for  this  week?  '  '  *  '  '  '  ' — 


Coiranent: 


7-    Do  you  feel  you  learned  a  lot^  this  j"    j  Q]]  jT^j 


Comment; 


8.     Is  the  Institute  paying  enough  I     j  j     j  f    j         T  1 

attention  to  your  important  problems?  ^  L — I  I — J 

Comment: 


9- 


Would  your  professional  peers  back  (     j  |     j  r~T| 

home  be  interested  in  what  you  L — •  L-J  I  !  l_J 

learned  this  week? 

Comment : 


10,     Do  you  think  you  could  teach  another         |     |  I  1 

group  what  you  have  learned  this  week?      LJ  I  ■  J  1  J  L-J 


Comment: 


Yes      Yes- "but"        No    I  don't  know 


II,     Is  the  Institute  meeting  your 
personal  expectations  for  it? 


□  □  □  □ 


Conanent : 


12,  Have  the  staff  failed  to  change 
or  correct  certain  matters  that 
were  brought  to  their  attention? 


□  □  □  □ 


Comment: 


13.     Are  you  getting  the  chance  to  study 
as  much  as  you  would  like  to? 


□  □  □  □ 


Comment: 


14.     Ars  you  getting  sufficient  oppor-            I     I          j     1          j    ^1  1  j 

tunity  to  socialize  and  recreate?  L — I  »  1  I  5  L. — I 


Comment: 


15*    Was  the  content  for  this  week                   |     I          j    ^          F   I  I  I 

challenging  and  important?  ^ — J  t — J  I  1  L_J 


Comment: 


16.    Are  you  working  hard  enough? 


□  □  □  □ 

Comment; 


Did  this  week  '^ang  together"  as 
an  instructional  whole  for  you? 


Yes'      Yes -"but"       No    I  don't  know. 

□  □  □  □ 

Comment: 


Would'  you  want  to  teach  another 
group  what  you  have  learned  this 
week? 


□  □  □  □ 


Comment ; 


Should  the  presentation  for  l.his 
week  have  been  preceded  with  more 
readings ,  discussion,  background 
materials? 


□  □  □  □ 


Comment : 


Would  you  recommend  including  all 
or  part  of  this  week  in  a  package 
to  be  used  in  summer  institutes 
elsewhere? 


□  □  □  □ 


Commenf'. 


If  you  have  a  message  for  the  people  running  this  Institute,  please  write  it 
here: 


SUM>1ARY  OPJNIONNAIRE 


CIRCE 
Surnmcr  1970 


5-DIGIT  CODE 

DIFxECTlONS:     Throughout  this  Institute  you  have  told  us  how  you  feel  about  a 
number  of  aspects  of  the  instructional  program  of  this  Institute.     Now  we  are 
asking  for  soiTie  OVERAI.L  reactions.     Check  one  response  category  for  each  item 
and  comment  if  you  wish  to  clarify  your  response. 

Please  use  -he  same  5-digit  code  on  this  OpinionnairG  that  you  have  been  usincr 
throughout  the  Institute. 

1.     To  what  extent  do  you  feel  the  following  objectives  were  achieved: 

Very      Fairly      Mini-  Not 
Well        Well        mally    At  All 

a.  An  understanding  of  the  problems  inherent 
in  the  management  of  an  educationaf  eval- 
uation project .  01  Q  Q  G3 


A  knowledge  of  methods  and  procedures  for 
evaluating  those  curriculum  changes  with 
which  you  as  an  Institute  participant  are 
concerned . 


□      □      □  □ 


An  awareness  of  the  major  alternatives 
available  in  terms  of  educational  research 

methcdologies-.  |— |  D  Q 

An  understanding  of  the  relationship  of 
eval.uation  to  the  problems  associated 
with  the  development,  dissemination,  and 
adoption  process  in  educational  innova- 

□     □      □  □ 

A^familiarity  with  communications  tech- 
niques applicable  to  proper  implementation 
of  the  decis?.i;>n-making  process  at  various 

levels  of  the  educational  system.   ^  [j  Q         •  Q  |~| 
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2.     You  probably  had  some  personal  objectives  which  w^^re  not  the  same  as  chose  of 
the  Institute.     If  so,  would  you  indicate  below  (briefly,  please)  what  they 
were  and  how  ^^'ill   they  were  achieved. 

Very      Fairly      Mini-  Not 
Well        Well        mally     At  All 


□  □  □  □ 


□     D     □  □ 


— — —  — — -     ...    □     □     □  □ 

3.    For  each  of  the  four  major  sections  of  this  Institute,  indicate  whether  time 
allocated  was  too  little,  about  right,  or  too  much: 


Too 
Little 

About 
Right 

Too 
Much 

a. 

Cook:     One  week  was- - 

□ 

□ 

b. 

Walbesser:     Two  weeks  was-- 

□ 

□ 

□ 

c . 

Stuf f lebeam:     Two  weeks "T^s^^-^^ — 

□ 

□ 

□ 

d. 

Asher:    One  week  was  — 

□ 

□ 

n 

4.     For  each  of  the  four  major  sections  of  the  Institute,   indicate  by  checking 
the  appropriate  box  the  extent  to  which  you  think  the  material  presented 


during  that  section  was  relevant  to  your  own  activities  or 


concerns ; 


Ve  ry 
Relevant 

Somewhat 
Re  levant 

b  iignt ly 
Relevant 

Not 
Relevant 

a.     Section  One  (Cook) 

n 

1  1 

n 

LJ 

1 — 1 

LJ 

b.     Section  Two  (Walbesser) 

■  □ 

□ 

□ 

□ 

Section  Three  (Stuff lebean) 

a  ■ 

□ 

□ 

□ 

d.     Section  Four  (Asher) 

□ 

□ 

□ 

□ 

the  following  items,  please  mark  the  appropriate 

box,  using  these  a 

IternativGS : 

SA  =  strongly  agree 

A  =  agree 

U  undecided 

D  =  disagree 
SD  =  strongly  disagree 

5.     The  content  of  this  Institute  has  direct  practical 


utility  in  my  institution. 

Coiranent : 


SA 


U 


D  3D 


□  □  □  □  □ 


6,     In  general,  the  approaches  presented  here  are  not  SA       A        U       D  SD 

feasible  in  my  institution.  Q     Q     |~j     Q  Q 


CoTument : 


ERIC 
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1 


ERIC 


7.     The  benefit  I  deri^/ed  from  the  Institute  was  nou  SA        A        n        o  SD 

worth  my  expenditure  of  time.  Q      Q  Q  Q 


Corament : 


Does 
Not 


8.     This  Institute  compared  favorably ,  overall,  with  SA      A      U      D    SD  Apply 

other  institutes  I  have  attended.  Q  D    D   CD  D 


Comment : 


9.     The  diversity  in  content  and  approach  presented 
by  the  consultants  was: 


SA        A        U        D  SD 

a.    Appropriate  to  the  objectives  of  the  Institute.      Q     til     CIl     D  Ql 


Coinment: 


b.    Satisfactory  to  me  personally.  -SA        A        U        D  SD 

□  □  a  □  □ 


Continent 


c»    Too  great  for  tne  to  be  able  to  assimilate  and         SA        A       U        D  SD 
integrate  things  as  well  as  I  would  have  liked.      Q  Q     [~|  Q 

■Comment: 
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10.     If  this  Institute  were  to  be  offered  again,   I  would      SA        A        u        D  SB 
recoramend  to  others  like  myself  that  they  attend  it.    □     Q     Q     [f]  Q] 

Corament : 


11.     Consultants  were  generally  not  available  for 
questio:.^3  and  discussion  outside  of  the  times 
scheduled  for  formal  presentations. 


Coinment  : 


SA        A        U        D  SD 

□  □  □  □  □ 


12,     After  pdrticipating  in  the  Institute,  I  feel  more 
competent: 


making  strategies. 


SA 

.  A 

U 

D 

SD 

evaluation  studies 

□ 

□ 

□ 

□ 

□ 

SA 

A 

u 

D 

SD 

research  projects 

□ 

□ 

□ 

O 

LJ 

sound  decision- 

SA 

A 

u 

D 

SD 

□ 

□ 

□ 

□ 

□ 
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13. 


Fo/  each  of  the  following  descriptive  statements,   indicate  by  checking  the 
appropriate  box(es)  which  of  the  .ejections  of  the  Institute  it  applies  to. 
The  following  code  is  used  for  the  column  headings: 

(1)  =  Project  management  models  and  techniques  (Cook) 

(2)  -  Behavioral  objecti^         design  of  assessment  tasks.,  and  construction 

of  learning  seque^    .s  (Walbesser  et  al . ) 

(3)  -  Evaluation  models  and  techniques ,  seminars  on  problematic  Simula^  ■ 

tions   (Stufflebeam,  Cuba,  and  Hammond) 

(4)  ^  Problems  associated  with  developing,  disseminating,  and  adopting 

educational  innovations   (Asher  and  Heisley) 


This  material  could  have  considerable 
applicability  to  my  work  at  my  own 
institution. 

I  really  don't. see  how  this  could  apply  to 
the  problems  with  which  I  have  to  work. 

It's  likely  that  I  will  be  able  to  put 
this  to  good  use. 

I  don't  see  how  this  material  could  be  included 
in  another  Institute  in  the  absence  of  the 
consultant(s)  who  presented  it  here. 

This  material  would  be  readily  transferrable 
to  other  Institutes  as  an  ins tructional 
package,  without  requiring  that  the  consult- 
ant (s)  who  presented  it  here  be  involved. 

I  would  recommend  that  my  colleagues  attend 
a  Workshop  or  Institute  which  had  this  topic 
as  its  central  focus. 


(1)  ^  (2);  (3)  C4) 


-7- 


14. 


following  aspects  of  the  Institute? 
nse , ) 


Excel 
lent 


_0,.K,  decided 


Inad- 

equate  Poor 


Facilities 

Time  schedule 

Budget 

Format 

Materials 

Equipment 

Staff  (AI  Elvell,  Mo  Oliver,  Ev  Barnes) 
Consultants 

Participant  Observers 

(Charles  Williamson,  Stuart  Pickard) 

Participants 


Food 


-8- 


15.     K-hich  of  Che  following  best  describes  your  overall  reaction  to  the  Institute? 


□ 


Very  Favorable  Undecided  Unfavorable  Very 
Favorable 


Unfavorable 


16.     A  six-week  time  allocation  for  an  Institute  like  this 


one  is: 


Much  too        A  little  About 
long  too  long  right 


A  little  Much  too 
too  short  short 


17, 


Do  you  have  any  message  for  the  staff,  consultants,  or  evaluators  of  this 
institute: 


ERIC 


PARTICIPANT  OBSERVER'S  DAILY  LOG  iNHUI 


I.    Facilitators : 

even t s ,  peop 1 e ,  de cis  i ons  ~ - 

Arrangements  that  had  a  positive  effect  on  some  facet  of  the  Institute. 


II.  Ii^ibitors: 

events,  people,  decisions-- 

Arranger^ients  that  had  a  negative  effect  on  some  facet  of  the  Institute. 


III.    Participants '  quotes : 

Quotable  quotes  that  reveal  participants'  views,  goals ,  satisfactions , 
dis appointments ,  etc . 


IV,    Human  Relations: 

Especially  along  the  dimensions  of  age,  sex,  race^  and  lodging  location. 


•V  1^ 


I?oTITUTK  CTAVF  SITK-v  iiHT 


i  i  1  i,  V  I.  .i  u  U  1-  O  ■ 


Date 


^Problems  solved • 


=<<Pi'obiems  porsifeting: 


^Principal  intent  of  UNHI : 


*Freebee : 


ERIC 


H.E.IV.  RESEARCH  INSTITUTE 
Self-Assessment  Profile 

*'No  one,  not^  even  an  expert,  should  evaluate  the  work  of  someone  else. 
In  fact,  thte  teacher  should  not  evaluate  the  work  of  tne  students;  they 
should  do  the  evaluating  themselves.     If  not,  v/hy  enjjhasize  the  objective 
of  self-direction,  which  obviously  includes  t^e  process  of  etraluatinn 
one  »s  performance  in  the  light  of  goals  that  have  been  self-determined? 
The  evaluation  function  of  the  teacher  is  to  provide  situations  whereby 
the  students  are  led  to  evaluate  their  own  procedures  in  the  light  of 
criteria  they  themselves  have  formulated. 


The  present  self-assessment  function  and  associated  profile  provide  a  situation 
whereby  you  are  being  asked  to  evaluate  your  professional  growth  in  the  light  of 
criteria  you  yourself  have  formulated  as  a  participant  in  the  1970  USOE  Title 
IV  HEW -Research  Institute.    Several  proba:ble:  criteria  of  professional  growth 
could  perhaps  have  been  identified  for  you  by  the  Institute  staff,  consultants, 
or  evaluators  but  '»not  even  an  expert  should  evaluate  the  work  of  someone  else.'' 
For  each  of  the  factors  that  you  identify  please  narrate  your  self-assessment,  of 
professional/growth.    Finally,  assign  yourself  an  overal 

the  two  courses  in  which  a  formal  assessment  must  be  reported  for  the  convenience 
of  the  University  Registrar  (the  UNH  grading  system  description  card  is  attached 
for  your  interpretive  needs) . 


Criterion  A: 


i 
t 


Criterion  B: 


Criterion  C: 


FQRf-lAL  ACADEMIC  GRADE  EVALUATION 


*  Education  881  (4  credits)  -  Methods  and  Tecimiques  of  Educational  Research  

This  course  is  a  critical  study  of  the 
principal  methods  employed  in  the 
investigation  of  educational  problems 
and  an  evaluation  of  the  procedures 
and  standards  ijsed  in  reporting  the 
findings . 

*  Education  897  f2  credits)       Research  Problems  in  Education,  >  

Tliis  course  is  concerned  with  the  (CIRCLE  ONE) 

individual  investigation  of  a  problem 

in  the  area  of  educational  research.  IIONORS  =  A     HIGH  PASS  =  PA^S  =  3 

The  course  is  designed  first  to  expose 

the  trainees  to  a  variety  of  educational 

research  efforts.    Each  trainee  will 

be  expected  to  develop  a  research 

proposal  which  will  have  practical 

importance  and  relevance  to  his  home 

district,  agency/  institution  or  othnr 

educational  organization. 


ADDITIONAL  COfflENTS: 


(CIRCLE  ONE) 
HONORS  =  A      HIGH  PASS  «  PASS  =  B 


ERiC 


Participant  Signature : 


Participant  Follow-Up  Evaluation  QuekStionnaire 
H.E.W.  Summer  Research  Institutes  1967-1970 
U.S.  Office  of  Education 
National  Center  for  Educational  Research  and  Development 


Prepared  by 

Bureau  of  Educational  Research  and  Tea  ting  Services 

P.O.  Box  Q 
University  of  New  Haaqashire 
Durham^.  N.H. 


March  4,  1971 


Full  Name 


Home  Address 


Business  Address 
Job  Title 


Participant  Personal  Data 


Home  Phone : 


Business  Phone : 


Year  of  Institute  Attended 


Years  in  Present  Position 


Directions 

1.  Although  the  Participant  Personal  batti  section  isn't  absolutely  vital,  if 
you  have  no  objections  to  it  please  fill  in  the  proper  blanks  listed  above, 

2.  If  ladivldual  questions  or  parts. of  questions  do  not  apply  to  the  Institute 
you  attended,  simply  omit  those  questions  and/or  parts. 

3.  When  asked  to  express  an  opinion  feel  free  to  use  the  back  of  the  questionnaire 
or  attach  a  separate  sheet.    PLEASE,  if  you  do  this,  be  sure  to  identify  by 
number,  the  question  you  are  responding  to. 

Be  as  SPECIFIC  and  PRECISE  in  your  responses  as  you  can. 

5.    Question  10  asks  for  two  responses  to  each  flection: 

(a)  the  importance  of  the  material  to  you  when  presented  at  the  Institute 

(b)  the  importance  of  the  material  to  you  now  in  your  present  job. 


ERIC 


1.  How  many  programs /projects  proposals  did  you  Initiate/write  prior  tp 
.  attending  your  institute? 

2.  How  many  programs/project  proposals  did  you  initiate/write  since  attending 
;       your  Institute? 

3.  Since  attending  the  Institute  have  you  written/initiated  any  of  the  following: 
Proposals  (Fed*,  State,  Local)  titles 


Evaluations ; 


Research  Studies; 


4*      My  job  role  and/ or  function,  since  attending  the  Institute  has  changed  from 

■       .. .  ' '  ■  ' '      -  :to'    ■       '  -  ■  ■     :  - ".-  :> 

5.      What  is  the  present  status  of  the  program/project/proposal  you  were  required 
to  submit  as  part  of  your  Institute  experience? 


6.      Since  your  Institute  experience  have  you  been  in  contact  with: 

A)  Institute  Staff  YES  WO  B)  Institute  Consultants  YES  NO 
C)    Other  Participants      YES  NO 

If  yes,  please  specify  the  context (s)  ;ln  which  this  contact  was  made 


ERIC- 


Slnce^ at tending  the  Institute  have  you  been  called  on  to  conduct  in-service 
training  programe.,  workshops  or  siaply  give  presentations  in  areas  the 
Institute  concentrated  on  '  «»  uue 

NO  NOT  APPLICABLE      (If  yes ,  please  specif y  the 

number,  content  and  audience  of  such  activities) 


Have  you  taken  or  been  involved  in  courses,  seminars,  workshops,  etc. /  since 
your  Institute,  related  to  educational  research  and  evaluation? 

YES  NO 

If  yes,  please  specify  date,  title,  place  etc. 


VJhat^have^you  made  of  the  instructional  materials  presented  to  you  at  your 
i.ns  titutG?  ' 


Please  rate  the  following  Instructional  areas/  ast  presented  In  your  Instlt^^ 
ACCORDING  TO  THEIR  IMPORTANCE:   (a) attending  the  InstltuL  and  * 
(o)     to  your  present  job  position. 


Developing 
and  writ- 
ing 

behavioral 
objectives 


The  con- 
struction 
and  identi- 
fication of 
learning 
heir archies 

Under- 
standing the 
various 
Gwdmlnistra- 
tlve  approach 
t:o;  program/ 
O  -f  V'..:: • 

ERJC  3ment 


Extremely 
Important 


Important 


IlLnliadlly      Not  Important      No  Opinion. 

Important      At  All  Response 


The  skill 
acquired 
in  util- 
izing the 
Program 
Evalu2i:ion 
Review 
Technique 
(PERT) 


Extremelv  Minimally      Not  Important 

Important    .Important    .Important      At  All 


The  CIPP 

model  of 

Evaluation 

(Context, 

Input , 

Process, 

Product) 


The  skills 
md 

practice 
acquired 
by  writ- 
ing 

computer 
urograms 


The  skills 
and 

practice 
acquired 
in  operat- 
ing 

computers 
and  computer 
supportive 
hardware 


-The...;,- 

statistical 
skills 

offered  as  an 
•''^♦-roductbry 
base  or 

general  revie^ 
of  those 
needed  In 
research 


No  Opinion 
Response 


Knowledge 
gained  from 
In t  erac tlon 
with  fellow  / 
participants 


Extremely 


Minimally     Not  Important     No-  Opinion 


Knowledge 
;  gained 

f rom  inter- 
;  action  with 

B.EvR.T.S,  1 
\\;stafffc: 
^:  (exclusive 
"Of  outside 

consultants) 

a 
b 

1 

\- 

•mm  MiM«M«M  *   II  1  1  1      MM  ■  1    1  ■■  « 

i.mporcant 

1 

i— — — — 

,  At  All 

1        -   .          ■  ■  ■ '  . 

Kesponse 
— ~— ~- 

Knowledge 
gained 
from  inter- 
action with 
Institute 
consultants 
outside  of 
the  class- 
room 

a 
b 

 .  

— — „  

•• 

■  . 

• 

,  ■ 

—  .  . 

Knowledge 
of  proposal 
writing 
skills 
acquired 

a 
b 

 — . 

-.-M««.  ^  ;:w — 

—  ^  : 

Importance 
of  readings 
in  required 
texts  as 
well  as; 
outside 
sources 

a 
b 

.                     :   '  i 

■  i 

11. 


12, 


I 


Do  you  feel  that  your  job  function  or  role  has  changed  aia  a  result  of  your 
participating  in  the  Institute? 

U     very  significantly      £7     significantly        £7    some  what 
£7     not  at  all        fj  .  don't  know 

Would  you  conunit  yourself  to  a  number  of  such  Institutes  (6-weeks)  to 
receive  an  advanced  degree  such  as  a  Master's  in  Educational  Research 
and/or  Education? 


/~7  YES 


/  /  NO 


/  /MAYBE 


Participants  to  Institutes  of  this  type  should  be  grouped  by  experience, 
job  function  and  expertise  on  a  more 

homogeneous  basis       /~J    heterogeneous        £7   no  opinion 


outstandins ;  : 

:  /T -no  opinion 


16.    What  do  you  believe  the  greatest  strengths  of  such  Institutes 


are ; 


I?.    What  do  you  believe  the  greatest  weaknesses  of  such  Institutes 


are : 


Additional  comment  or  personal  opinion: 


Complete  any  statements  on  back 


I 


Question  # 


6A 


1967 

1968 
1969 
1970 


TOTAL 


6 
8 


YES 


(42.9%) 
(88.9%) 
10  (47.6%) 
13  (56.5%) 


37  (52.2%) 


8 
1 


NO 


(57.1%) 
(11.1%) 
11  (52.4%) 
10  (43.5%) 


30  (40.8%) 


6B 


1967 
1968 
1969 
1970 


TOTAL 


YES 

4  (28.6%) 

6  (66.7%) 

7  (33.3%) 
16  (69.6%) 

33  (49.3%) 


NO  . 

10  (71.4%) 

3  (33.3%) 

14  (66.7%) 

7  (30.4%) 

34  (50.7%) 


6C 


1967 
1968 
1969 
1970 


TOTAL 


YES 

8  (57.1%) 

7  (77.8%) 

14  (66.7%) 

22  (9  5.  7%) 

51  (76 .1%) 


6 
2 
7 
1 


NO 

(42.9%) 
(22.2%) 
(33.3%) 
(4.3%) 


16  (23.9%) 


YES 

NO 

1967 

3 

(21.4%) 

11 

(78. 

6%) 

1968 

3 

(33.3%) 

5 

(55. 

6%) 

1969 

11 

(52.4%) 

9 

(42. 

9%) 

1970  ~ 

14 

(60.9%) 

9 

(39. 

1%) 

TOTAL 

31 

(46.3%) 

34 

(50. 

7%) 

YES 

1967 

4 

(28.6%) 

10 

1968 

4 

(44.4%) 

5 

1969 

7 

(33.3%) 

14 

1970 

12 

(52.2%) 

11 

ERIC 


TOTAL 


27  (40,3%) 


N/A 


1  (11-1%) 
1  (4.8%) 


2  (  3.0%) 


NO 


(55.6%) 
(66.7%) 
(47.8%) 


40  (59.7%) 


TABLE  10  A 


During  the  Institute 


1967 
1968 
1969 
1970 
TOTAL 


1967 
1968 
1969 
1970 
TOTAL 


Extremely 
Important 
2-14.3% 

19- 


1.-11.1% 

8-38.0% 
5-21 .7% 
16-23.9% 


^tremely 
Important 
3-21.4% 
3-33.3% 
8-38.0% 
8-34.9% 


Important 


8-57.2% 
6-66.7% 

11-  52.4% 

12-  52.2% 
37-55.2% 


Minimally 
Important 


Not 
Important 


1-7.1% 
1-11.1% 

3-13.0% 
5-7-.5% 


Present  Position 


Important 


5—35 . 8% 
3-33.3% 

9-  42.9% 

10-  43.5% 
27-40.4% 


Minimally 
Important 


1-  7.1% 

2-  22.3% 
1-4.8% 
3 ~ 13.0% 
7-10.4% 


1-  4.8% 

2-  8.7% 

3-  4.4%, 


■'Not 
Important 


1-  7.1% 

2-  9. S% 
1-4,3% 
4-6.0% 


No 
Opinion 


3-21.4% 

1-11:1% 

1-4.8% 

1-4.4% 

6-9.0% 


No 
Opinion 


4-28,6% 
1-11.1% 
1-4.8% 
1-4.3% 
7-10.4% 


TABLE  10  B 


During  the  Institute 


1967 
1968 
1969 
1970 
TOTAL 


1967 
1968 
1969 
1970 
TOTAL 


Extremely 

Minimally 

Not 

„„-No-- 

Important 

Important 

Important 

Important 

Opinion 

9-64.3% 

2-14.3% 

3-21.4% 

1-11.1% 

4-44.4% 

1-11.1% 

1-11.1% 

2-22.3% 

3-14.3% 

11-52.4% 

5-23; 8% 

1-4.8% 

1-4.7% 

3-13.0% 

11-47.8% 

8-34.9% 

.1-4.3% 

7-10.4% 

35-52.2% 

16-23.9% 

^  3-4.5% 

6-9.0% 

Present 

Position 

Extremely 

Minimally 

:  .\Not' 

No  ■■ 

Important 

Important 

Important 

Important 

Opinion 

1-7.1% 

6-42.9% 

1-7.1% 

2-14.3% 

4-28.6% 

1-11.1% 

1-11'.1%  ■ 

3-33.4% 

2-22,2% 

2-22.2% 

8-38.1% 

9-42.9% 

3-14.3% 

1-4.7% 

3-13.0% 

6-26.1% 

10-43.5% 

3-13.0% 

1-4.4% 

5-7.5% 

21-31.3% 

23-34.3% 

10-14.9% 

8^12.0% 

TABLE  10  C 


During  the  Institute 


1967 
1968 
1969 
1970 
TOTAL 


1967 
1968 
1969 
1970 
TOTAL 


Extremely 
•Emportant 

2-  14.3% 

3-  33.3% 
5-23.8% 
7-30.4% 
17-25.4% 


Extremely 

Important 

4-28.6% 

2-22.2% 

4-19.0% 

10-43.5% 

20-29.9% 


Important 


8-57Tl%^ 
2-22.2% 
12-57.1% 
15-65.2% 
37-55.2% 


Minimally 
Important 


2-14.3% 

2-  22.2% 

3-  14.3% 
1-4.4% 
8-11.9% 


Present  Position 


Important 


•28. W 
2-22.2% 
11-52.4% 
11-47.8% 
28-41.8% 


Minimally 
Important 


■14.3%" 

1-  11.1% 
4-19.0% 

2-  8.7% 
9-13.4% 


Not 

Tmportant 


Not 
Important 


1-  7.1% 

2-  22.2% 
1-4.8% 

4-6. 0% 


No 

Opinion 


2-14.3% 
2-22:3% 
1-4.8% 

5-7.5% 


No 
Opinion 


3-21.4% 
2-22.3% 
1-4.8% 

6-8.9% 


,^table::io.^d.-^.-.^--  — — , :  . 


During  the  Institute 


1967 
1968 
1969 
1970 
TOTAL 


1967 
1968 
1969 
1970 
TOTAL 


Extremely 
Important 

1-  7.1% 
3-33.3% 

2-  9.5% 
8-34.8% 
14-20.9% 


Extremely 
Important 


1-  7.T%^ 

2-  22.2% 
4-19.0% 
4-17.4% 
,11-16.4% 


Important 

7-50.0% 

3-33.3% 

11-52.4% 

9-39.1% 

30-44.8% 


Minimally 
Important 


1-  7.1% 

2-  22.3% 
5-23.8% 
5-21.7% 
13-19.4% 


Present  Position 


Important 


3-  21.4% 
2-22.2% 

4-  19.0% 
12-52.2% 
21-31.3% 


Minimally 
Important 


3-  21.4% 

4-  44.4% 
8-38.1% 
7-30.4% 
22-32.8% 


:  Not 

Important 


1-4.8% 

1-  4.4% 

2-  3.0% 


Not 
Important 


1-7.1% 

3-  14.3% 

4-  6. 0% 


No 

Opinion 


5-3578^ 

1-  11.1% 

2-  9.5% 

8-11.9% 


No 
Opinion 


6-43.0% 

1-  11.2% 

2-  9.6% 

9-13.4% 


ERIC 


TABLE  10  E 


During  the  Institute 


Extremely 

Minimally 

Not 

No  . 

1967 

Important 

Important 

Imiioi'taTit 

IMUKJi  t  till  L 

upinxon 

5-35.7% 

3-21.4% 

6-42.9% 

1968 

1-11.1% 

5-55.6% 

2-77  2% 

i96f 

2-9.5% 

12-57.2% 

5-23.8%  ; 

2-9.5% 

9-39.2% 

13-56.5% 

1-4.3%  : 

TOTAL  / 

12-17.9% 

35-52.2% 

10-14.9% 

1-1.5% 

:  9-13.5% 

Present 

Position 

Extremely 

Minimally 

Not 

No 

1967 

Important 

Important 

Important 

Important 

Opinion 

2-14 .3%  : 

1-7  IS; 

1968 

.  1-11.1% 

2-22.2% 

3-33.4% 

2-22.2% 

1-11.1% 

1969 

5-23.8% 

,  7-33.3% 

3-14.3% 

3-14.3% 

3-14.3% 

1970 

11-47.8% 

12-52:2% 

TOTAL 

17-25.4% 

23-34.3% 

11-16.4% 

6-9:0% 

10-14.9% 

************************************** 

*9t*-k***-kie-k-k**  ********* 

TABLE 

10  F 

During  the  Institute 

Extremely 

Minimally 

:  Not:;  ^ 

Important 

Important 

Important 

Important 

Opinion 

1967 

2-14.3% 

6-42.9% 

4-28.6% 

2-14:2% 

1968 

4-44.4% 

1-11.1% 

2-22.2% 

•2-22.3% 

1969 

2-9.5% 

10-47.6% 

,7-33.3%  . 

2-9.6% 

1970  • 

1-4.3% 

4-17.4% 

3-13:0% 

3-13.0% 

12-52.3% 

TOTAL 

■  ■    ■  f;  - 

9-13.4% 

2i-31.-3% 

16-23:9% 

14-21.0% 

Preserit  Position 

Extremely 

Importajfit 

Minimally 

, ' ,.  .Not,,. , 

No 

Important 

Important 

Important 

Opinion 

1967 

1-7.1% 

3-21.4%; 

/  4-28.6% 

5-35:7% 

1-7.2% 

1968 

2-22.2% 

■  3-33.,3%?t^ 

2-22.2% 

2-22.3% 

1969 

1-4.8% 

6-28. 6%1 

6-28.6% 

6-28.6% 

2-9. 4%r-;.-' 

1970  . 

2-8.7% 

'•2-8:7% 'r.:,'' 

5-21.7% 

2-8.7% 

12-52.2% 

TOTAL 

6-9.0% 

14-20.9% 

17-25.4% 

15-22.4% 

15-22.3% 

ERIC 


TABLE  10  G 


During  the  Institute 


Extremely 

Minimally 

Not 

No 

iyo7 

Important 

Important 

Important 

Important 

Opinion 

1-7. 1% 

6-42.9% 

5-35.7% 

1-7.1% 

1-7.1% 

1968 

2-22.2% 

"3-33.3% 

3-33.3% 

1-11.2% 

1969 

,  

7-33.3% 

10-47.6% 

2-9.5%  . 

2-9.6% 

1970 

2-8.7% 

5-21. 7% 

4-17.4% 

12-52. 2% 

TOTAL 

3-4.5% 

18-26.9% 

23-34.3% 

8-12.0% 

15-22  3% 

Present  Position 

Extremely 

Minimally 

Not 

No 

■:;9!',7  '■ 

Important 

Important 

Important 

Important 

Opinion 

4-28.6% 

4-28.6% 

5-35.7% 

1-7.1% 

1968 

2-22.2% 

4-44.4% 

2-22.2% 

1-11.2% 

1969 

6-28.6% 

7-33.3% 

6-28.6% 

2-9.5% 

1970 

2-8.7% 

4-17.4% 

4-17.4% 

13-56 . 5% 

rOTAL 

2-3,0% 

16-23.9% 

17-25.3% 

16-23.9% 

16-23.9% 

»*** tilt********* *****liiiicliic**ic1i-^1c***it***1x **********  *^ 

TABLE 

10  H 

During  the  Institute 

Extremely 

Minimally 

; 'Not.'  , 

No  : 

Important 

.Important 

Important 

Important 

Opinion 

1967 

5-35.7% 

8-57.1% 

1-7.2% 

1968 

4-44.4% 

3-33.3% 

2-22v3% 

1969 

2-9.5% 

11-52.4% 

5-23.8% 

1-4.8% 

2-9.5% 

ir*70 

2-8.7% 

6-26.1% 

2-8.7% 

4-17.4% 

9-39.1% 

TOVU 

13-19.4% 

28-41.8% 

10-14.9% 

5-7.5% 

11-16.4% 

Present  Position 

Extremely 

Minimally 

Not  \ 

•  v;:; -No 

Important 

Important 

Important 

Important 

Opinion 

19  7 

3-21.4% 

9-64.3% 

1-7.1%. 

1-7.2%  ; 

ises  .  ■'• 

3-33.3% 

/2-22 ;  2% 

4-44.5% 

mr 

3-14.3% 

9-42.9% 

6-28.6% 

1-4.8% 

2-9.4% 

197) 

.  4-17v4% 

6-26.1% 

2-8,7% 

1-4.3% 

10-43.5% 

TOTAL 

13-19.4% 

26-38.8% 

12-17.9% 

3-4.5%  . 

13-19.4% 

I 

I 

I 


I 
I 


TABLE  10  I 


During  the  Institute 


Extremely 

Minimally 

Not 

No 

Important 

Important 

Important 

Important 

Opinion 

li967 

6-42.9% 

7-50.0% 

.1-7.1% 

1968 

2-22.2% 

5-55.6% 

2-22 . 2% 

;,.  . — 

,  -  -  ■.; 

1969 

15-71 .4% 

5-23.8% 

1-4.8% 

1970 

10-43.5% 

8-34.8% 

5-21.7% 

TOTAL 

33-49-3% 

25-37.3% 

8-11.9% 

1-1.5% 

Present 

Positic)n 

Extremely 

Minimally 

M  M  i«> 

Not 

No 

Important 

Important 

Important 

Importsmt 

•  *  -      ■  -  -  .  ■ — 

Opinion 

1967 

4-28.6% 

8-57.1% 

1-7.1%  • 

1-7.2% 

1968 

2-22.2% 

2-22.2% 

3-33.4% 

2-22.2% 

1969 

14-66.7% 

5-23.8% 

1-4.8% 

1-4.7% 

1970 

9-39.1% 

9-39.1% 

5-21.8% 

TOTAL 

29-43.3% 

24-35.8% 

10-14.9% 

2-3.0% 

2-3.0% 

**************************** *********^ 

TABLE 

10  J 

During  the  Institute 

Extremely 

Minimally 

■  Not  -  .  • 

No 

Important 

Important 

Important 

Important 

Opinion 

1967 

4-28.6%  : 

9-64.3% 

1-7.1% 

1968 

•  4-44.4% 

4-44.5% 

1-11.1% 

1969 

9-42.9%  ;V' 

11-S2.4'!- 

1-4.7% 

1970 

8-34.8% 

8-'34.8% 

4-17.4% 

3-13.0% 

TOTAL 

25-37.3% 

32-47.8% 

5-7.5%  . 

3-4.5% 

2-2.9% 

Present  Position 

Extremely 

Minimally 

-.Not  ;' 

No 

Important 

Important 

Important 

Important 

Opinion 

1967 

,2-14.3% 

9-64.3% 

1-7.1% 

2-14.3% 

1968 

3-33.3% 

2-22.2% 

4-44.5% 

1969 

7-33.3% 

7-33.3% 

5-23.8% 

1-4.8% 

1-4.8% 

1970 

8-34.8% 

6-26.1% 

6-26.1% 

3-13.0% 

TOTAli 

20-29.9% 

23-35 .8% 

15-22 .4% 

5-7,5% 

3-4.4% 

ERIC 
I 


TABLE  10  K 


During  the  Institute 


Extremely 

Minimally 

Not 

No 

1967 

Important 

Iffiportant^ 

Important 

Important 

Opinion 

1-7.1% 

7-50.0% 

3-21.4% 

2-14.3% 

1-7.2% 

1968 

4-44.4% 

3-33.3% 

1-11.1% 

1-11  2% 

1969 

3-14.3% 

13-69.0% 

3-14.3% 

1-4  RS; 

.;::?«,!»  ■■ 

1970 

6-26  1% 

12-52 

X  **  — ,  *J  <U  •  "J 

1     A  /Ift 
X  -*f  .  h'o 

A  -  H  .  f  * 

TOTAL 

10-14  9% 

19-17  QS; 

D  —  /  .  0  ^ 

Extasentely 

Minimally 

Not 

No 

1967 

Impissrtant 

Important 

Important 

Important 

Opinion 

6-42.9% 

3-21.4%  . 

2-14.3% 

3-21.4% 

1968 

l-.lj!!..l%) 

4-44.5% 

4-44.4% 

1969 

2-9.5% 

11-52.4% 

3-14.3% 

4-19.0% 

1-4.8% 

1970 

8-34.8% 

9-39.1% 

4-17.4% 

1-4.4% 

l-.4;4% 

TOTAL  . 

11-16.4% 

26-38.8% 

14-20.9% 

11-16.4% 

5-7.5% 

*ie*'iMt***il****itik^lk  ********* 

TABLE 

■::io-L, 

iDuring  the  Institute 

jExtrsBely 

Minimally 

Not 

.  No 

Impojcfearit 

Important 

Important 

Important 

Opinion 

1967 

'  3-2L..4% 

11-78.6% 

1968 

;  :5-SSi;6%' 

2-22.2% 

1-11.1% 

1-11.1% 

1969 

12-57.1% 

1-4.8% 

1-4.8% 

1-4.8% 

1970 

9-39.1% 

4-17.4% 

1-4.4% 

TOTAL 

23-34-3% 

34-50.7% 

6-9.0% 

3-4.5% 

1-1.5% 

Present 

Position 

ExtXEsniely 

Minimally 

:  Not 

No 

ImpoTiliant 

Important 

Important 

Important 

Opinion 

1967 

3-21-aa% 

6-42.9% 

3-21.4% 

1-7.1% 

1-7.1% 

196« 

2-22-2% 

4-44...  4% 

3-33.3% 

19i9  '  .  ■ 

9-42.1% 

7-331.3% 

3-14.3% 

1-4.8% 

1-4.8% 

1970 

12-52i2% 

9-39;.  1% 

1-4.4% 

1-4.4% 

TOTAL 

26-38.8% 

10-14.9% 

3-4.5% 

2-3.0% 

TABLE  10  M 


During  the  Institute 


1967 
1968 
1969 
1970 
TOTAL 


1967 
1968 
1969 
1970 
TOTAL 


Extremely 
Important 

3-  21.4% 
1-11.1%. 
7-33.3% 

4-  17.4% 
15-22.4% 


Extremely 
Important 
2-14.3% 


7-33.3% 
3-13.0% 
12-17.9% 


Important 


7-50.0% 
5-55.5% 
9-42.9% 
11-47. 8% 
32-47.8% 


Minimally: 
Important 


4-2876¥ 

3-  33.3% 

4-  19.0% 
7-30.4% 
18-26.9% 


Present  Position 


Important 

7-50.0% 

S-55.6% 

7-33.3% 

12-52.2% 

31-46.3% 


Minimal  ly- 
Important 


Not 
Important 


4-28.6% 

4-  44.4% 

5-  23.8% 

6-  26.1% 
19-28.4% 


1-4.4% 
1U.S% 


Not 
Important 


3-4.8% 

2-  8.7% 

3-  4.. 5% 


No 
Opinion 


1-4.8% 
1-1.5% 


No 

Opinion 


1-7.1% 

1-  4.8% 

2-  3.0% 


************************* 


*******************  *****^j,^^^^^..jj^ 


Very 

QUESTION  Significant 

11  1967  2  (14.3%) 

1968  3  (33.3%) 

1969  4  (19.0%) 

1970  4  (17.4%) 

TOTAL  13  (19.4%) 


Significant 


Somewhat 
Significant 


Not  At  All     Don't  Know 


7 

(50.0%) 

4 

(28.6%) 

1 

(  7.1%) 

4 

(44.4%) 

1 

(11.1%) 

1 

(11.1%) 

10 

(47.6%) 

6 

(28.6%) 

1 

(4.8%) 

10 

(43.5%) 

5 

(21.7%) 

3 

(13 .0%) 

1 

(  4.4%) 

20 

(29.9%) 

22 

(32.8%) 

9 

(13.4%) 

3 

(4.5%) 

12 


1967 
1968 
1969 
1970 


TOTAL 


YES  , 

NO 

MAYBE 

6 

(42.9%) 

2  (14.2%) 

6 

(42.9%) 

6 

(66.6%) 

2  (22.2%) 

1 

(11.1%) 

11 

(52.4%) 

2  (  9.5%) 

8 

(38.1%) 

10 

(43.5%) 

6  (26.1%) 

7 

(30.4%) 

33 

(49.3%) 

12  (17.9%) 

22 

(32.8%) 

13  1967 
1968 
1969 
1970 


TOTAL 


HOMOGENEOUS 

HETEROGENEOUS 

NO 

OPINION 

5 

(35.7%) 

6  (42.9%) 

3 

(21.4%) 

4 

(44,4%) 

3  (33.3%) 

2 

(22.2%) 

2 

(9.5%) 

9  (42.9%) 

10 

(14.9%) 

15 

(65.2%) 

6  (26;1%) 

1 

(  8.7%) 

26 

(38.8%) 

24  (35.8%) 

17 

(25.4%) 

14 


1967 
1968 
1969 
1970 


Extreme 
Value 

5  (35.7%) 
2  (22.2%) 
8  (38.1%) 
13  (56.5%) 


Handled  by 

Less 
Well  Known 

9  (64.3%) 

5  (55.6%) 

10  (47.6%) 

7  (30.4%) 


No 

Significant 
'  Value 


2  (22.2%) 
2  (  9.5%) 
1  (  4.4%) 


No 
Opinion 


1  (4.! 

2  (8.7%) 


TOTAL       28  (41.8%) 


31  (46.3%) 


5  (7.5%) 


3  (4.5%) 
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FILMED  FROM  BEST  AVAILABLE  COPY 


OM»S  IGH1_  HVALUA'XIQN 

Ji*  -Vill  enhancs  the  "espertiae**  of  the  participanta. 
B.  Will  establish  a  degree  of  rapport  between  the  1 970 

participantp  the  Institute  p0rsor4nal,  and  tho  LEA. 
0*  It  also  involves  ths  LEA  directly  with  the  institute 
personnel.  (** I  know  that  they  care*"?o 

1.  ISA  personnel  and  participanto  ^ifill  b©  able 
to  give  a  candid ^  yet  valuable,  evaluation 
of  the  program »  which  in  turn  will  ba  a  direct 
indication  of  the  affcctivenoss  of  the 
Inatituto  upon  the  ininiediate  needs  of  ths  agancy 
(i.e.)  Promotions,  contributions  of  Participants 
 -etc.  '  V  ' 

2 •  Infonssation  can  be  obtained  by  the  institute 
leaders  and  HSV  as  to  tlie  direction  of  future 
■  ■  -institutes'. 

3.  Uoet  icportantr  LEA's,  participants,  and  future 
participants  will  feel  a  part  of  th©  "Powor 
s true turo**  and  srlll  be  willing  to  assist  and 
participate  in  the  prograss  icor©  estonsively* 

Do  Will  enhaiice  the  personal  rQlatlonship  batwoen  all 
of  tha  bureacracyo 

omsniomAim  and  coi?FSR5r?cs.  c^u-s 

A.    QuQstioiinaire  to  participants 
B»    Questionnaire  to  supervisor  or  somQ  superior. 
1.  Problems  Ecay  ba  encountered, 
(a*)  Lack  of  responae 

(b«  y  A  breakdown  in  cotmunications  b© two ©n 
participant*  suparviscr*  and  institute • 


(c.)  **R©d  tape"  of  bsauracracy  and 


C.  Follow-up  by  a  coni>rence  call  >rith  teas  ciembers,  " 

D.  Ccnferenee  calls  to  Eai-ticipahxs 

!•  Problens  to  ba  ancounteredr 
(a.)  "Tru«  QTaluation 

!•  Unaware  of  potential 

-  ior  ••Uncle  Ped".) 

3»  She  laclT  of  confidentiality  of 

long  distajioe  teiephona  calls. 


:"Poople, '.attend  Ir^ntituton  I'or' a  variety,  of  reasonss 

:  (a.).  Vihat  wa3  yo\:r  p./orai^oior'^ai  :.-oa:j.on(3)  ?-  ,  ••■ 

.-.  (b>}  V/nat         y-ur  ;^..ij:T  personal  roa3cn(G)  ?  ^  •  • 

■■Vrhat'  problem  cituationi.  :^.id  you  oii^^oun'Oer 5; 

■  (a» }  ■Rolca.30'  ■  . 

-  ^o)  Transportation 
.(c  ■.Fii\anc:c:i; 

What  v;as  o::peatod  of  you  after  .your  return  froia  the  Inst  itu  to: 
=  ;,(a«.)  .frc.^  your  employer 
:  (b".')  'frcm  yourL3elf  ■■■  ■ 

■  ■.(c*)'  .irora  yGur:rcun.]J.y 

:  ^' ."   ■    ■  ■  ■ . ■  '.■■■■.^  •  ■  .■    ■"       ,■"  ^. Q-''.. 

-V/hat  are  ..the  slzillz^  and  Lnov;lodgo-  you  pc-rccive-  as  .cs^ontial?.' 
:  V/hat  i3  the  condition  of  school  iritcsration  in  your  area  ? 
Hcv/  is  it  af fee-bins  "^''^^clc  ocaohers  ai-'id  blade  administratcr^s  ? 
How.  is  it  affect in^  you  p^rocnally  ? 

Some  peplo  believe  that  degrees  of  integration  between  whites 
and  blacks  are  essential  ac  Irnov/lcdge.  is  diaperaed  frcn  a 
center  of  understanding  such  as  the  Univeraity  of  now  Hciupshire 
throuf^h  this  inetitutop    Vihile  niere  oentacta  'betvieon  diverse^'' 
socking  ccnirnon  knowledge  are  not  a  panacea  for  our  social 
illsa^    She  abcence  of  contacts  blocks  even  the  potential  for 
understanding*     I  an  interested  in  the  experlGnoos  of  social 
inter-action  you  have  had  v;ith  southern  and  northern  whites 
and  oou thorn  and  iiorthern  blacks  in  various  regions  and  under 
varying  circumstances* 


A*  \ihat  arc  they.' 

B  ,  H  :/iV  ar  o  It;h  e y  ^  a o  o  t  e  i 

10i»     ]jO  you  ti.ii'i      j    i.a;.-..or  iw.vt/.iv! . j- i;oj.-pj."ai  to  you? 

il.  ^  How  vloc::  yvuv  sur^c-.- v.L3or     g  j.I '  ci"!:. :.  joar  .^\/L.;..r.  ii':^;;;;  tl:o 
Ino'tituta  ? 

13.     Has  t-ioro  -^oo-^j  a  i-'  ■/ — J-^^^^       -OoiCii  or  roio  as  a 

rc3v.lt.  Ox  y-^y^yc  pi:,:' .iclpat  Ion  j.n  t^  j  I.ij'oltutc  ? 

13"   ■  VJ'ould  you  "^.j  Qt''.c:iJ  th^  Ii..:3tituto  a^_^;j.in  :^"or  si:;  ./o^Ics 

(nO  ^^^or^3  or  Lc::j  ? 

i-r;.     v;::>uld  ycu  re\..^rn        you,  r^:,^i.':vGvi  ::i  Uc;,^rc;.  L.asxers 
or  -^-actora'U:  in  L.:/aca    onal  .lie ii^^arcri  7 

r"  ■    ■        :  ■■ 

15*     IiC:VO  you  vmttQn  any  yrcp:3aio  or  dir^:ctcd .  :::.v:y  pro^}oots 
other  than  the  rcquirod  pr^^osals  ?  ■ 
(a« )  FeclGral 
('b«)  State 
(c. )  Local 

16-    Kov/  usefi;l  has  the  Institute  inf crrLriticn  boon  in  your  jol)  ? 

17c    Kas  yoiir  supervisor  irivitocl  you  to  share  your  experiences 
with  others  ? 


PAirr  I.I 


1»      liovV  long  (lid  it  tj..l{e  you  1  :>       i'to   .our  •      ...j-^n  ? 


ycur  ,jropo.r:al  b/.^ni  . 
(a)  iinplcmeirood  -  ■  ^ 

lb)  ?iri.ancoi 

(c)  accGotei  in  your  area 
3*       Si'.!  you  Lave  Gifi'iaulty  ,^o;j-:Inr  \;h::  y.'v„;rcir:     lurided  ? 

5"       ^^Lao  do  you  iJc^l  a:-::  j^ui  .faster:;  o:^  your  proposal  ? 


?«    /  V.  iiici  .•,r..struc ticTi  araa  ".-r-  rj  .ao3"';  :;-;:lr)2'iri  to  you.  in  the  clcvolop- 
fucnt  ox  your  ^^i^opooal 

3.      Have  you  rcGiiL;v.,a  a;iy        i  rtanco  iro~i> 
(a)  Inn titute  atafi 
(tj)  participants 
■(c)  Ccnsultcintc 
.   (d)  Other. 

9»      What  is  your  basic  need  in  the  fuifiilircnt  of  your  pro j act  ? 

10. •  The  Observation  of  the  project/fir  any  consultative  assictanco 
tliat  the  participant  r.ay  ha"'G> 
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•  I  1 1 .  V  «,/  w 


What 


■  •  J..  ^       ^  a       w  w 


1    i       uil  Li  J.  i 


.   i  .  ik,  M  tin.*  U  S--  -   -  S/  ^>         ■  * 


'•yd 


9,      ..hen  v;:i3  hi.:/l':cr 


10,     GivQ  aj-.y  ?.ddi-ti-c:.: 
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■   .  ■  r     Q  n 


clifTer.  ::c.    in    thn   r  e  svo::.-^ .   of   the  rarrici.an.a 
^^^^   rea.on    -o-   t  h  e  ir  ,    L  t  endan  c  e   to   the  In.titu.e. 

was-  readily  reco-iiiz--d   th--^    f ^   >>^v.  r      •    .  • 

-..^u    L...^.    u.;e  T>e>r.on   nan   tne  cnanca  to 

^^^^^-^  ad/anta^e   of   the  vopportun  i  t;- . 
The.avera^^e  resnon.e   ..s   .eared   tcard   their   .ersonal  reason 
as   beinn  their ' .rofe. sional   reason.      Tne   personal   reason  in 

most   cases  va^s   ar^scci^ter   v^^-'^   -'■n-.-;..  > 

.--o^x.^beu   ..^..i    -.aeir   per.::onal  reason. 

^-^  •     a .       eleas  e  0 
Tra-nsport  at  ion  3 
c .      Finances  '  8 

Release  was   not   a  problerr,  because  all  of  the   persons  vere  not 
eraployed   during  the  suniner. 

Transportation  was   a  problen  for   three  of  the   participant s , 
because  Of  their  mode  of   transportation.      Airline  schedales' 
difficult   to  rr^ect   and  their   livin.,  a  considerable  distance 
from  the  airport. 

Finance.,  was   i^rtant  to  t  h  e   in  dividual  s  .  b  ec  au  s  e  most'  of  these 
persoris   had  to  readjust  their  lives  to  the   stipend.  However, 
the  participants  were  reinbursed  for  their  travel  expenses,  the 
najor: problem  was   centered  around  the  money  earned  during  the'  ■ 


•rirlcyer  I? 


•      i>  ^  h  n,  V  i  o  r  r..,..   o\)  ;^  o  c  t  i  v  e 

Learn  inn  ^oir arc  hies  ? 

T>  r  o  r  r  'i    /   r  o  ,i  e  c  t 

"anan;enent  1 

CIP?  ^  3 
CorTiputer  Pro.^^rans 

Research   Techriiciues  1 
Social  Interaction 
Proposal  V/ritinf-^   Skills  5 

5 .     Excellent     .  0 

Very  Good  2 

Good  .  3 

Fair  3 

Poor         '  ;  0 

*^Not   important  2 


These  persons  lived  in  a  bl ack  c ommunity .  Their  schools  are 
not   involved   in  the  problems. 


.■■  u  ;  ,    :  r  t  : •    rezv^o::\ :}  e  v 
'  "■    not      .-it    it     h  o  u  I  cl  b 


C5  1 


ri  c  :  i  0   1   i  i"i  t  e ; :  r  a  t  i  o  n 


black   t  eac 'ler  s 


1  1 


y  IT-:  -  ret  it  ive 


"Ic  r-o'bl'6:r: 


■'■'zi-^^or  able 
;n favorable 
Indifferent 


5 
1 
3 


8. 


B 1  a  c  k  s 


Whites 


Job 

For  supervision 

Under  supervision 

Social  Interaction' 

Professional 
. Activities 


southern     ?J  o  r  1 1 1  e  r  n 

3 


8 


Southern  Northern 


3 
k 


9  •     Direct  Decision- 

.  2  ■ 


8  : 
6 

Indirect  Decision, 


N/A 


8 


7 


Most  of  the  decisions  are  made  throu,^h  the   supervision.  Though 
there     are  two   individuals  who  make   decision  directly,   they  also 
attempt   influence  their  supervisor  and  nake  decisions   for  their 
subordinates. 
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•Ml  of   the    prir  t  i  c  i  r^wi        inaUr'U'e      thai    t;.cir'   :ioci. '-i  .'ni:  Jxr- 
u  T) u a  1 1  y  n.c  c  ^^^.u t  ed^ . 

IC.      Hee   Iter;  !•  ^ 

All  of  tiiC  partio  irants    felt   thax   thr?   r^ro-^ra-i  va::;   helirful    to  ^hen 

11.  :  All  of  the  respondents  indicated  that  their  :.;u  i:ervi  sors  felt 
that  the   pronre.n  was   helpful   for  their  personnel. 

12.  Promotion  Demotion  i^'o  chan'3;e 

2  1  6 

13.  /II  . persons   v/ould  be   happy  to   attend   tiie   i^ro'^rani  arain. 
Fornoreorlesstirae. 

ih »      All  responses  was  yes. 

15.        V/ritten  Direct ed  Pro  j  ect  -  N/A 

■    k  1  1  ^ 

Other   participants   (workshop,    etc.)  .  *. 

a.  Federal  1 

b.  State  2  . 

c .  Local  6 


16.  Useful  Not  useful  N/A 

8  0  1 

17.  Share  Experience  Not   share  experience 

U  ^.  ■                                 .  5 


All  of  tiU3  par  t  i  c  i  r-^-n  s  i^r^j  red  t.  ii^ir  r  er  i  e  n  c  e  *  vi'i;  c-tri'-^rs 
thro  a/:'.!   converGatlonn  ,    {^t:.  . 

Tart   i:  .        ,     .  . 

1.      The   iiv  er?.r.e  '  ti7\e    Tor  vr  i  t  inr/  t  h  6^  r^rorosal^.;  *  was    four  (h) 

v;eekc.      Other   joo   corir:)  i  t  tr.  or.  t  s   cause  nost   of   the   vers  on  3  t.\e 


delay    in   return  inn   their    nrojosal^  to   IJ ,  h' ,  :i  , 

•2,      Irnpl  eaent  ed  0 

Financed  1 

Accepted   in  your   area  *V 

Negotiatinr,  2 

Pro-'oosal   inconrlete  2 


3.      Difficulty  No   difficulty  K/Resp. 

T  0  2 

h.      Accept  inr:  Rejecting?  N/p 

6  1  2 

5*      Educational   innovation  2 

Projected   ideas    ( n ev /r exi s ed )  '  3 

Community  concern  2 

Educational  concern.  1 

N/ Response  1 


6.  Financin:?5  7 
Complet  inf^  2 

7.  '     See   Item  i| . 

ERIC 


:  or 


.11  r  e  s  :  o  n  c  0  : 


.  ^3  s  I   tr  an  c  e  m 


!')•      Indicated   in  liarritive   :  erort . 


1.      All   resnon^es   were   excollont  (8) 

Sane  as   above  (3) 
3.     yeG    (  no   (  ?  ) 

Helpful      [P.)  [  ' 

3.     yes  (8) 

The   averarre   resronse  -   3  years 
6  .     yes       3  r  e  3  p  o  n  3  c  s 
T  .       yes   ( 5 )  no   ( 3 ) 

^.     yes        (B)  ' 

All  of   the   promotion  depended  on   the  openin.p;.      Only  two 
supervi:3or5    indicated  a  specific   job.      They  vere  rro^notions 
to   super vi  cscr  s  , 
•     Supervisor  1' 
■Principal  1  ' 

Asst.    Principal  1 

Pronot ion  ,  v:/ o  chanp;e  of  job   or   cater;-ory  3 


EKLC 


^o.theru  Jla--:3  a;id  .i  ,  • "     ti  tut  e   1  070 

.  ^^^^i^.  reaction  t.)  t-^  i^vnitTt-:^  d--:rt-^   i-v- 

years  was,  if  tfeeyVrA^'.^o;,..^',^         ■':!,,:-;:^-;!^';--V'  ^T'-  ^^■'S't  "^^ree 

systenis.  ■     '    :  -  ^  J-.,-rv^;.itea  school 

Advanced  dagree3  for  ooj.thern  bi  pc-s  b^^v-  tr^-.-^'^  ti -....n ,  i 
.     :naster's  de~rees T'^f pi^r  y^V  t'^e^-.,J-^  ;^^-^c.:  .nptitutions  conferred  " 

the  ade.uac.  or  ^.e .r^^^d^raduaLl?--  Uo^^-^S^  J^Led, 
programs  -..ith  -nore  stringent  residence  Te^:;^^ el ei.t^  ^^^^ 

Bu.ces3fu'■f■f■'e':';^f^';■'''^'''"''f'°^'  °*=taoles,  a  Southern  black 
lUgSereLi-'tlon:  f  =rogra:r„  he  would  tend  to  move  Into 

a,„te;-?i  I"-!^f  ""'-?f-^"al  '3aae,  blacks  are  most  Uicely  to  =rss=a  t 
.  fuSer  InstftSesf"  "'^-'^^  --dvanoed  degree,  butr.ssed  pe?haps>,lth"  * 

Der'iol  fni^?'.'l"?  'h-^:  initial  rationale  for  a  weelc's  orientation 
peiiod  xox  tne  ooutnern.  black  participants  in  the  lOVr  Instit-a'e:°  -^ 

e°clfi??p^'''n?h^^j  ""''^^  aciuainted  With  ' 

2.     To  caiovi  them 

opportunity  to  re'^atp  to  rpcii  rioi^+  i^i .  . 
and^see^the  ^^l^o,  ,e. 

'^r^^'re^S'^'^f^^  informal  gatherings ,  to  express 

ooncerno  relating  to  the  program  and  their  own  situation. 

ERIC 


c.-ncnr::3  r.j  1  i.- •   v,-  -r^^  .i'-rn-  ^w^;]  the5r  o-ri  situation. 


-.,  v  .-J    f '  ■;•  '  i"'        4,    ■  „      ,         ~r      ^         ^   f    4.  4. 


•■^/^'^  'ieeds  avid  abilities  of  bi'ic  :;art  ic i  oants . 


)i   ":^r^i;e^i:"  -Lnat  'oizen  occurs  in 

all 


7.     To  e^'t-^bLlsh  r-.  social  cl  i  ..a     .  u 'lat  would  .:.a.:i.::i:^a  Icarniap 
^nd  , -::.rti;U  inbe— ■crion  ^;".d  oh^rla;. 

Dhe  fc":  ^..-v-rln-  ^''exioio  schedule  w;as  ::/^nned. 

^    First  day:   Wednesday  a^'ternoon,  arrival  and.  settlings  in" 

,ved:i-?3d-:iy  ev^alrif?;,   first  orientatica  ^z'^oion  at 

jtoaes  liall        dirscusslon  of  .J6*.v  aaf;iaad  and 

lew  .liin^land  life,  with  3_  ecial  rei-rcace,  to  yev;  aa;:]; 

jecond  day:  Taursday,  breakfast  at  the  ..urvvci.d  .loaejt-ad, 
li.aphaois  oa  rrettiii^^  ac  j  :.aiat 

ihur^.   :,ft^rn:.:.n      ..act^rr:  on  ::a.;i:a.3    rith  ' 
Irs.  Ada-ns    ,    -      .  .  .  ' 

i\  Joaa:;on,  ad  uii^s i -n^  jfllcer  for  biaol  stadonts 

Dlscai^cica  of  blaclv  stuCdorits  oa  ca:npU3, 

availing  ffi-^ting  "Tillh  blacl,  .iG..iborG  of  tao  v/ider 
co'n  unity/recreatloa/theater  etc, 

Third  day:     Friday  iTiornlng,  Individual  Interviews  with  Curwaod- 

^^rid;ay  axteracon;  lie^tla^j;  v/ith  lloli  Institute 
staff  :ne:nbGrG 

3vealug  free  : 

Pourth  day:  Saturday,  raorainj ;  Individu.al  Intervieifs. 

Saturday  afternoon:     Area  sight-seeing. 

Saturday  evening:  BE.^TS,  institute  staff  and 
participants  ;^icnic  and  suiyier  at  hone  of  Jan 
and  ilerb  oohelbel 

Fli.th  day:     Sunday  norning :  Ohurch  --siglti^seelng,  beach,  etc, 
and  afternoon  . 

avsninr^.:     -/elcorr^ras  of  other  nembers  of  the 
institute. 
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The  Initial     "  ans  fur  or:.ent,a:^i.)n  ;vere  .:*everly  :..odified. 

1/     The. lateness  of  recrulflng  earl/  showed  that  a  full  work 

five  day  ;=eriod,  st'^^xtj,  ng  on  .Vednesday  v;ould  include 
fourth  of  July  holid£.y, 

2.     In  s::=it9  of  the  later  starting  date,  all  of  the  part- 
icipants did  not  arrive  on  time  and  sone  did  not  arrive 
vrlthln  the  disir7nated  nsriod  vfnen  individual  intervievrs 
were  scheduled.     Only  six  of  the  ten  blac;.c  riartici ';.ants 
vfere  intervievred. 

•  .     3.     Two  hlac:  participants,  not  ::^art  of  the  ten,  and  one 
paticipant  iro;a  Guan  Participated  in  the  full  five 
day  ^eriod. 

Wednesday  evening  four  persons  :net  vrith  me, 

Thursday  :-nornins,  five  cane  for  breakfast.     This  was  a  very 
pleasant  experience.     ^'i.lio  knov/s  whom'^  is  an  accepted  and  important 
dislgnation  in  Southern  black  groups  that  are  criss-crossed  by 
colleges  attended,  ::iutual  friends,  /jinship '  ani  fraternal- 
ties,      3everal  bonds  of  solidarity  were  forged,     Tnfor;aatlon  ^^o^^ 
liew  ::;ngland  and  IJew  rlanipshire  was  given  and  mf or:Tiation  about 
participants  was  shared  informally, 

Friday's  day,  awaiting  the  arrival  of  more  .nenibers,  favored 
several  inforfual  relationships  with  Institutional  staff  members. 
3y  afternoon,  I  v/as  able  to  intervievf  two  niern-bers.     Sat.  rday  iriorning 
I  interviewed  two  more.     The  picnic  supper  x-^as  held  at  ny  :-:lace. 
A  friendly  cooperative  spirit  prevailed  as  the  five  black  members  , 
were  joined  not  only  by  he  Institute  and  BERTS  staff  members,  but 
also  by  other  early  arriving  vfhite  participants.     I  rejoined  the 
group  Sunday  afternoon  in  time  to  v:elcome  more  black  participants 
and  to  interview  two  more,>     During  the  institute,  I  laet  the  rest 
of  the  black  participants,  took  some  sight-seeing  in  Portsmouth, 
but  I  did  not  have  an  opportunity  to  complete  the  interviewing. 


List:     potential  guests. 

Lena  Coleman:     Long  time  resident  of  Ooncord,  M.H. 

Melvin  3olden:     Deck  Road,  Loudon,  N.H.     Politically  active 

in  the  Democratic  party. 

The  3acon  Brothers:     of  Pembroke^i  oTOer  of  a  rug  cleaning  plant 

in  Concord,  N.H.  Mrs.  Bacon  works  for  the 
Family  Service  Agency  in  Concord. 

Clifford  Lav/rence  of  Andover,  Mass.:    works  with  the  Deaf  and 

Blind  Children's  Division  in  Concord. 
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The  questionairs  that  the  -  ortic"  ••'•.nt<-'  -r*    -a'    .    :  .  .  , 

ground  inicr:;iati on  obo^-t  f-in     .-.r-ti  ^   ^  -•-^o-^'^ 

Ground  and  the    oc,s^  b< e  '-^h^  il^.ood^  y  -:-^^^^ 

adult,  ensased  in  .duration: — -euu..  ...nal  o.c...-round  of  an 

The  interview  schedule,  v.ust  be  vlev/ed  as  a   TeU-.lnarV  for-n 
probing  at  generalised  areas  of  ooncsrn  that  I  thou.^ht    t^ht  be  -el- 
ev,ant  to  help  us  understand  the  interv-' ew^--e' -•ur-' n.-  'X  "nn^"?^-^  ■  + 

no.  aotua.,./  ..n    situation  a.:  a  base  line  for  later  eo-ii-arisons. 

The  questions,  I  believe,  are  relatively  self-explanatory.  ■ 

■-->t-nli^^f??--ocS:->.2'^  social  inter?.,  i     ■  .■  <  •     recorded  has 

;....i.enoj.a lor  uiit-iU  ines ,   out  needs  an  i..-,  ,  :    .  ,.  •  - 


ihe  six  interviews  car:e  fro::-  four  sta^     .    '  Tvro  fro^i  "'ennessee 
uwo  iro:n  .ouisiana  (north  -  south),  and  one.--  -n  fro.a    iss^Sst  -'^l  ' 
v-a-  nor-;r-r(pr^  of  the  interviewees  were  :3ales,  three  fe^alesl'  '  Qm 
o^er"f?f-       f        J^'  ^el^^^en  the  ages  of  42  and  47.  One 

a^e  °Is  42      '  "  under -30.    To- round  it  out  the  average 


All  lntervie;,ees  are  .basically  stabiles.     All  are  livin^  in  the 
state  where  they  were  born;  all  received  under.;raduate  educatioS  in 

I't  e  s5^?e      '^'?H^"if  °'         2^°^-^         graduate  study  in  thei? 
hcie  stcite.     uf  the  five  vrao  were  -Carried,  four  of  their  spouses 
were  also  born  and  educated  in  the  home  state.  spouses 

All  but  one  held  master's  degrees;  four  of  them  in  education,  one  in 
.nistory      One  husband  completed  high  school  and  one  colleo-e  ah 
three  wives  have  college  degrees  -  one  a  masters'.  o  . 

A.,  total  of  9  children  •belong  to  the  five  who  a..re  married;  one  has^ 
xour  children,  one  teo  and  the  rest  have  only  one  child. 

The  interviewees  represented  a  rather  immobile  grouD  of  uersors  - 
well  educated  within  their  respective  communities.     The  men  ^re  married 
to  college  graduate  wives.     The  family  size-,  with  one  exceotion, "the 
farm  family,  is  very  small,'  but  rather  tyoical  of  the  southern  " 
middle  Class  Negro  family,  long  represented  in  "school  teaching". 

The  educational  achievement  of  parents,  in  general,  is  of  a. 
lesser  level  than  that  of  their  children.     Only  two  of  the  inter- 
viewees had  fathers  who  had  completed  college;  only  one  had  a  mother 
who  had  completed  college.     One  had  a  mother  who  attended  colleo-e 
for  owo  years,  but  whose  father  had  comDleted  high  school.  The*^ 
education  of  the  other  four  fathers  was  7th  -  8th  (2),  4th  (1)     DK  (1) 
Of  the  other  four  mothers,  one  completed  high  school,  (2)  7th  -  8th  ' 
and  one  DK.  xa, 
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-r-.^^  nt,  ^_.-c....^.o      •ax-.x.wS,   oaree  i:.ri,:iers  ana  two  .nothers 

'•"-xe  D..  s,   uwo  ia^ners  and  three  nothers  were  7th  ^^rade;  the  "cb^^dren" 

nave  aade  hl^  leaps  oj^pared  with  th-i ir  parents.  '"^''^ 

|1     All  but  one  of  the  intervie'wees  saw  thc-aselves  as  having  been  recrait 
for  tne  pr^ra:a  oj  a  blao        The  >3in,-le  one  directly  attributab{p  to  a 
wnite  recruiter  ca:.ne  aoout  as  an  accident.     (I  i.iistook  Texas'  \  i  m 
(predonmantly  v;nitej  for  Irairie  view  (predominantly  blacK}.) 

Two  to  three  contacts  -  rather  chain  effect  wer  used  to  recr-iU 
ine  process  was  :nuch  -.-lore  ti-.ie  oonsu-ins  than  I  had  anticipated,  but* 
,i  fee:   we  opened  up  nsw  channels  for  future  trainin,-.      -uch  of  what 
"nappens"  seorjs  not  to  filter  through  to  blac>  educators. 

Four  of  the  participants  .were  recruited  raainly  throun-h  college 
contacts;  two,  through  state  departments  qf  education.  ,  ° 

All  were  pleased  to  be  co.:iin~  to  the  Institute,     Per  the  -^ost 
•oart,  it  Mas  an  opening  up  of  an  unkno^.«i  section  of  the  country 
•  only  one  expressed  mixed  feelings  of  being  maneuvered  to  come  to 
used  as  a  synbol" 

#3    .-lost  cause  professionally  to  learn  and  st'^.rt  o^  continue  graduate 
v;orl-c  In  a  more  challenging  environment.     Several  were  encoura-'^ed  bv 
supervising  persons  to  come. 

Personally  most  had  a  curiosity  about  a  new  section  of  the 
country.     One  had  .nade  plans  for  his  family  to  share  in  the  experience 
the  last  weelc  -  joining  worx  and  recreation.  ' 

■■3^4    None,  expect  few  logistic  details, 

'/5    The  shortness  of  the  time  between  being  accented  and  having  to 
come  caused  the  most  problems  -  for  some  it  meant,  release  from  regular 
suamer  obligations,  for  one  an  actual  salary  loss  and  failure  to  complete 
one  more  course  for  certification  (9  I\Few  Ham^Dshire  credits  are  not  ' 
worth  anything)  -  in  this  case.     ?or  one  without  charge  cards,  the 
actual  money  for  travel  was  a  big,  hurdle.     The  trin  for  the  most  ' 
part  was  very  long  and  exhausting.     The  least  number  of  problems 
were  faced  by  those  nartlcioants  who  knew  in  Anril  that  they  would 
probably  come,  "  ■ 

■96  _^No  one  seeiried  to  have  a  clear  picture  of  what  the  exoerience 
would  mean  when  they  returned  i.e.  no  clear  responsibility  "to  tal^e  baclc" 
information.     One  iremtpp  had  relatlvPl  y  rl         r^spo srSflity  to  tge  ®     -  ^ 
extent  that  most  of  tRe^t^ae  Tir^om^3&e^  Institute,  he  should    ■ . 

nave  been  on  the  job.    He  came  with  the  blessings  of  an  active 
super.lutendent  who  could  forsee  benefits  to  the  system  from  this 

tS^tSe'tt'te  nf?iof  Ti^''''^^  raalcing  a  full  reDort.  :  ' 

^  aboul         oxner'iencr^°"  (blac.  sector)  and  to  tallc  to  local  ^ 
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— .^0  one  .anticipated  any  problems  when  they  returned  to  H...  ......  V  ^ 


..:  Only  one  -onber  je-.;,;(^cl  fuVly  aelf -as cured  about  b^-ln.-  In  the' 

■  P-^pdra:n-._     jne  covered  self  isuurance  or  anxiety  or  both  in  a  gicb--i 
Wi^S^fS^'-..^''  le:.:rn  iTh:rL  needed  to  b-  le'-rncd.     The  rest  expressed-  ■ 

.  ti^r:^X^^  .^ayc.  ana^ae.,::-.  s  str.,:i.:  ^inx'.otles,   principally  th^^t 

;2jVn-  r;?""  i^-!?  /•esultlns^fro:.:  n..rd  school  ye::r  ..itl.g  :ted' against 
^    anxlSly':^erl;f^-^i./^;"::;r;f  -.rtlc  V^an  t '  3 

■  loarn.  ~  "    '"^^  .         ^-^ •       ^  '--1  nere  vr5,l  ^ing  to 

.7^9     JnspeciflGa  roaaarcli  s-lils  and  co:nT)utor  i^-zl^  Is  w^re   >eneral^  y 

■  ^*  l;^^®        a- a  Inst  ae.     ueo-le  here-  .na/  be  okay  but  It's 

i-cd  .;ral  material  and'  the  next  person  •r.ight  not  be  okay".-  - 

b.  don't  uae  it  to  ae;.arate  blacKs  from  vr.ites 

c.  don';t  deisrade 

Several  .generalized  that  "that  Is  the  way  it  should  be."  I.e.  - 
...t6stln.»5  at  any     .me.     Overall,  they  viewed  the  testing  as  "-jartof  the 
SatJie"  part  of  the  price  they  vrouid  have  to  -^ay  to  be  In  the'  pro~ran 
(Resignation  to  fate!)  "  -  ' 

,^11     The  overall  picture  on  intergatlon  

12 

.     Jo:neti'.]ej3  a  black  teacher  Is  offered  only  one  job  -  if  she  refuses 
 outl 

•/hlte  teachers  are  resigning,   but  have  other  job  options  blacks 
do  not  possess, 

-If  lnterf;ratlon  Is  hapyenlns  at  all,   It  Is  to  replace  goodblack 
teachers  with  poor  -  less  qualified  v.^hlte  teachers.  iTon-tenure 
teachers  :nay  be   noved.   Theoretically  teachers  have  a  choice,  but 
actually  It's  a  Hobson ' 3  chpioe . . 

Five  were  quite  mindful  of  intergration  problems  and  relatively.  ■ 
knowledgeable  about  them  even  v/hen  basically  In  a  segregated  sit- 
uation.    Only  one  see.-ned  to  just  accept  the  status  quo. 

onp  i^if^^"^^;';^^^^^^^-^^  dominant  roles  over  whites,  but  onlv 
one  real  -  the  other  one  was  "paper  supervision."  ^  : 

■yi3    Integration  had  affec  ted  '  personally  only  twos 

was  Zvii/rrnf^'''"^  with  white  teachers  and  the-  other  one 
a  oSn      Pn?S  ^  pseudo-lntegrated  situation  to  an  all  blaC- 

'  ^'^e^e  very  aware  of  dlscrlalnatlon. 
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.^'14     Inter-racial  contacts.  In  .ruth 

1.    Trades  :  eo^le  anly .  -  •  whl  t  e  ----  It^nited  area 
ii-  •    ,;ide  e>:rerlences 

a.  .iiducational  ,  :neet Ings       vriiite  LlacK 

b.  .  aiac!.  friends  in  Boston  and  'assachuc.ctts 

c.  Integi'ated  church 

d.  studying  at  ^V/vhite'^  institution  .  '  . 

e.  Jchool  sind  Dolitical  contacts  (.noney  from  blaclc  teacher 
for  IJAACr )  .  ' 

Supervisor  in,  ar:ny  school  vride  contacts  —  blac.v  -  white 
6  vree\:.  institute  M.Y.  teacher  co-';'.ng  in  •       ,  . 

health  prorra:n  not  for  BLACKS 

v;  hite  grad  school  fan'ly  white 

In  general,   v/ider  contacts  v^ith  whites,  northern  blacics  see.iied 
to  depend  largely  on  out  of  souths         \  . 

K         Contacts  maintained  with  migrating  blaolcs.  . 

2,         Sufficient  ranlcing  in  educational  system  to  be 

allowed  to  attend  wide  range  of  professional  nieetings. 

5,         Broad Ing  experience 

a.  arny  attached  to  educati ^nal  unit.  , 

b.  graduate  study  in  vrtiite  institution  :  . 
4,         Job . experienc es  ... 

.  -  southern  whites. 


llovr  did  you  ii'^ar 


about  the  1 us 1 1 tu t e 


;^hat  v/as  your  inlti  li  reaction?. 

for  u  variety  of  reasons: 
i:  r  o  f  e  3  3 1 0  rial  r  e  a  so  n  ?  . 

,;.or30:ial  rea3on(s)  ?  • 

vHiat  Influonoe  did  personal iu:ed  coatacta  with  adrnlaistrator s 
of  the  Institute  have  on  your  deol:3lon? 

What  proble-i  situations  did'  you  encounter? 

a)  rele':;i3e  ■  . 

b)  transportation 
0 )  finances 

,v'hat  will  be  exi^ootod  of  you  after  your  return  from  the  Instituted 

a)  froi.i  your  e;:iployer? 

b)  frof:i  yourself? 

c)  froin  your  fa.nlly? 

Do  you  anticipate  any  .proble::is  vrhen  yo.;,  return: 

a)  worhing  relationships? 

b)  inter-p  ersonal  relatlonshi  c.s  ? 

What  anxieties  do  you  feel  about  the  adequacy  of • your  back- 
gx^ound  for  success  in  the  pro^jrain  as  you  nor  understand  It? 

jliat  are  the  s^cills  and  knowledge  you  now  perceive  as  essential? 

How  difficult  is  ix  for  you  to  vievr  a  testing  program  as  a 
diagnostic  tool  for  snail  g^^oup  planning  rather  than  as  a 
jiidgr.ent  as  personl  deficiency?  , 

ihat  Is  the  condition  of  schools  integrated  in  .your  area? 
Hovf  is  it  affecting  blac:c  teachers  and  blaok  adniinistrato^s  ? 
How  is  it  affecting  you. personally? 

■jo:ae  people  believe  that  degrees  of  intergatlon  between  whites  anc 
blacks  are  essential  as  knovrledge  is  dispersed  from  center  of 
unders-^'and  Inrr,  such  as  this  University  through  this  Institute, 
vfhile  Dere  contacts  betv/een  diverse  seeking  com-aon  knowledge  are-  r 
a  panacea  for  our  social  ills.     The  absence  of  contacts  blQcics  , 
even  the  ootenial  for  understanding.     I  am  interested  In  the 
experiences  of  social  inter-action  you  have  had  with  southei^n  and 
northern  whites  and  southern  and  northern  blacks  in  various  . 
regions  and  under  varying  circumstances. 


Peocle  ca:^e 

a)  "./hat  waj 

b)  :«liat  v/'vo 


to  Institu 
your  :::a jor 
y Dur  jor 


Pre-orientation  Questionnaire 

Dr.  Sarah  T.  Curwood 

1,  How  did  you  hear  about  the  institute? 

2.  What  was  your  initial  reaction? 

3.  People  cone  to  Institutes  for  a  variety  of  reasons: 

a)  What  was  your  major  professional  reason? 

b)  What  was  your  major  personal  reason (s)? 

4.  What  influence  did  personalized  contacts  with  administrators  of  the 
Institute  have  on  your  decision? 

5,  Vfhat  problem  situations  did  you  encounter? 

a)  release 

b)  transportation 

c)  finances 

6.  What  will  be  expected  of  you  after  your  return  from  the  Institute? 

a)  from  your  employer? 

b)  from  yourself? 

c)  from  your  family? 

7,  Do  you  anticipate  any  problems  when  you  return: 

a)  working  relationships? 

b)  inter-personal  relationships?  _  „     

8,  IVhat  anxieties  do  you  feel  about  the  adequacy  of  your  background  for 
success  in  the  program  as  you  now  understand  it? 

9*      What  are  the  skills  and  knowledge  you  now  perceive  as  essential? 

10.  How  difficult  is  it  for  you  to  view  a  testing  program  as  a  diagnostic 
tool  for  small  group  planning  rather  than  as  a  judgment  as  personal 
deficiency? 

11.  IVhat  is  tlie  condition  of  schools  integrated  in  your  area? 

12.  How  is  it  affecting  black  teachers  and  black  administrators? 

13.  How  is  it  affecting  you  personally? 

14.  Some  people  believe  that  degrees  of  integration  between  whites  and 
blacks  are  essential  as  knowledge  is  dispersed  from  center  of  under- 
standing such  as  this  University  through  this  Institute.     IVhi le  mere 
contacts  between  diverse  seeking  common  knowledge  are  not  a  panacea 
for  our  social  ills.    The  absence  of  contacts  blocks  even  the  potential 
for  understanding.     I  am  interested  in  the  experiences  of  social  inter- 
action you  have  had  with  southern  and  northern  whites  and  southern  and 
northern  blacks  in  various  regions  and  under  varying  circumstances. 
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instructicnal  effectiveness  of  the  1967-1970  ttCVr  Research  Institute 
programs  held  at  the  ilhiversity  of  New  Haacnpshire. 

Perhaps  of:  ^greater  concern  to  DHEW/OE,  hov/ever ,  is  that  the  base- 
line data  accrued  from  the  New  Hampshire  prograns  provide  an  interesting 
and,  in  many  respec^te  unique  set  of  ffactrors  that  warrant  careful  assess- 
ment.   An  in-depthi  evaluation  of  these  date  Sid  permi^  of 
some  of  the  assunigetd^cms  underlying  ilite 

Our  princ3^^1  concern  was  >  o£  c:cmrse  ,  to  assess  the  instructional 
effectiveness  of  r^esjearch  training  prograniis:  as  typified  by  the  four  New 
Hampshire  summer.  fiirslH: tut es.     But  the  natera'I  by-product  of  such  an  assess- 
ment is  the  identiScation  and  specificatEiijn^^  facti^sns  upon  which  recommenda- 
tions raiight  be  vmak^  :csa3itCTgm^aa:y  research  training  designs 
and  aMiernatiiK^e  .slascsse                pr ogmm: th at  might  be  incorpor- 
ated into  OEls:iEaritiisra:^ 

This  documeait   the  1970-7:1  mEl^  :R  Report  — 

is  focused  on  two  maijtor  evaluative  targ^d^::^     (1)  an  e.v;aluation  of  the  1970 
summer  institute  effort,  per  se,  aridl         s^tiiEongitudih^^  assessment  of 

the  instructional  aeSectiveness  o f  th e k 'l^giiSF^ 7 0  Institute  programs  separately 
and  in  the  aggrega^ev.    The  results  of: :l3ie£ssa assessments  include  not  only 
substantive  daca  batr:also  their  associait^eSl:  recommendations . 
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EVALUATION  DESIGN       :  IMPLEMEiNTATION  PROCEDURES 


The  HEIV  Institute  evaluation  effort  wsiszfocij^sj^  on  three 
principal  evaluative  targets  as  follows : 

EvaLuati'Ve  Target  #2 

To  assess  the  degree  of  success  of  itSne.- j]^70  Institute 
program  including  its  objectives ,  iansE^antional 
desijgT?;,  and  pre-instructional  impjlemenmsstxon 
procedures . 

Evcctuative  Target  #2 

To  assess  the  effectiveness  of  tSib: i^^i^^^^ 
prograin  as  implemented  during  tihie  l^^ 
training  program. 

Evaluat'ive  Target  HS 

To  conduct  a  longitudinal  f o  1 1  o w-rdt^ rteirafliaT^^ i on 
of  the  1970.  Institute  par ticipant^^i^EEsaS^^^^^^^ 
trainees  who- had  participated  in  piseaads^^ 
Hampshir^e.  Iiisi:itute  programs  as  weUIitL, 

Results  of  these  analytical/evaluative  axa3:^s[^t:i©s  are  to  be 
used  to  provide  baseline  data  with  regard  to  reconussp^SS^niDns  for  the 
design,  development  and  implementation  of  alternatSll^feHpssining  strategies  to 
be  reviewed  by  the  Research  Training  Branches       .Cem^^'Mor  Educational 
Research  and  Development \,  DHEIV/OE. 

The  responsibility  for  the  assessment"  of  tfer  1970  Institute 
Design  (Evaluative  Target  #1)  was  executed  by  the  external  evaluator  of 
the  Institute  program,  the  University  of  Illinois  V  Geiattosr  for  Instruc- 
tional iResearch  and  Curriculum  Evaluation  (CIRCE)  .    33i^^ripximary  methodology 
used  incixided  functional  system  analyses  of  the  administrative  components 
of  the  program.     Techniques  consisted  of  observation,  inquiry  and  interview. 
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together  with  the  standard  evaluative  instruments  deemed  pertinent  to  the  ■ 
design,  development  and  implementation  of  the  pre-ins tructional  segment  of 
the  Institute  effort. 

Analytical  :.nd  evaluative  activities  focusing  on  the  effectiveness 
of  the  instructional  program  itself  were  conducted  principally  by  the  BERTS 
staff  and  CIRCE  evaluators ,  respectively.     Host  evaluative  instruments 
were  designed,  developed  and  administered  by  BERTS  and  CIRCE  and  supplemented 
by  pre-designed  achievement  instruiiionts  provided  by  instructional  consultants. 
Jn  brief,  an  array  of  evaluative:  methodologies ,  techniques  and  instruments 
were  employed  in  the  assessment  activities  associated  with  Evaluative  Target 

The  longitudinal  follow-up  evaluation  effort  (Evaluative  Target  #3) 
was  conducted  by  the  Institute  staff  in  cooperation  vjitli  a  participant  observer 
of  the  1970  program  v/ho  served  as  a  field  cvaluator.     Due  to  restricted  funds 
available  for  a  comprehensive  on-site  evaluation  effort,  the  follow-up  activi- 
ties were  confined  to  mail -questionnaires  to  all  participants  (1967-70)  and  a 
select  se^mer"^        cn-site  visitations  of  1970  participants  by  the  BERTS 
staff  and  the  field  evaluator. 
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PRODUCT  EVALUATION 

The  Bureau  of  Educational  Research  and  Testing  Services,  a  sub- 
unit  of  the  Department  of  Education,:  University  of  New  Hampshire  conducted  a 
six  week  research  training  institute  during  the  period  o£  July  6  -  August 
14,     The  grant  provided  for  the  training  of  thirty  two  (32)  participants  to 
be  selected  from  any  of  the  fifty  states  in  the  UNION  and  its  Territories. 

This  institute  was  viewed  as  the  first  phase  of  a  five  year  develop- 
mental effort  in  the  area  of  educational  research  training.     It  was  proposed 
in  the  original  document  that  a  series  of  five  summer  institutes  would  be 
conducted  over  the  next  five  sunmiers,  starting  in  1970,  which  would  in  f act  ' 
be  viev^y';;i  ?v5  creating  a  series  of  basic  building  blocks  called  "instructional 
components .       These  ^^components''  would  be  researched  and  developed  until  they 
could  stand  alone  and  meet  the  objectives  stated  for  them.     Further,  these 
instructional  components  could  then  be  assembled  in  any  manner  that  a  research 
training  director  might  wish,  thereby  enabling  him  to  create  a  series  of  long 
or  short-term  institutes  depending  upon  his  specific  needs  and  set  of  objec- 
tives . 

The  general  objective  of  the  1970  summer  institute  was  to  conduct  a 
training  program  which  would  concern  itself  with  the  problems  of  training 
educators  at  the  doctoral ,  post-doctoral  level  in  the  skills  of  project  manage- 
ment, educational  research  and  evaluation  as  aids  in  the  process  of  decision 
making.     Under  this  general  obj ective ,  the  institute  set  for  itself  the 
following  seven  major  themes  with  their  specific  sub-sets  of  instructional 
objectives.  " 
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They  are  as  follows: 


The  first  major  theme  inll  be  the  study  of  the  problems  inherent 
in  the  management  of  an  educational  evaluation  project. 

Specific  objectives: 

At  the  end  of  the  institute  the  participants  will  demonstrate 
their  ability  to  perform  the  following  tasks: 

a)  Define  systems  analysis  and  the  listing  of  the  basic  steps 
related  to  systems  analysis  procedures. 

b)  Define  -the  meaning  of  management  information  systems  (MIS), 
list  the  major  component  pcwts  of  an  educational  informa- 
tional  management  system,  and  define  the  essential  "steps 
in  the  implementation  of  an  MIS  in  actual  practice. 

a)    Define  program,  planning,  budgeting  systems  (PPBS)  and 
list  the  component  parts  of  a  PPBS  system. 

d)  Define  Program  Evaluation  and  Review  Technique  (PERT).  List 
the  basic  elements  of  PERT  and  prepare  a  PERT  network  from  a 
simulated  problem  situation. 


The  second  major  theme  will  be  the  study  of  the  problems  inherent 
in  evaluating  the  particular  curriculum  changes  with  which  the 
trainee  is  concerned.    The  study  of  the  problem  of  evaluation  will 
be  attacked  by  having  the  trainees  identify,  write,  and  evaluate 
objectives  for  instructional  programs  in  the  area  of  curriculum  change 
in  which  they  are  interested.  '  • 

Specific  Objectives: 

At  the  end  of  the  institute  the  participants  will  demonstrate 
their  ability  li?  perform  the  following  tasks: 

a)  Identify  the  objectives  for  a  specific  program  in  the  area  , 
of  their  personal  interest. 

b)  Discriminate  between  well  written  or  poorly  written  behaviorally 
stated  objectives. 

a)    Identify  and  construct  a  set  of  performance  chjectives 

d)    To  translate  (where  possible)  into  performance- objectives 
stated  in  the  abs tract  that J tf^y  . prepared  as  a^^  for 
this  institute. 
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The  third  major  theme  u>ill  be  the  study  of  the  major  alternatives 
open  to  the  educator  in  terms  of  educational  research  methodologies. 
For  instance:    the  experimental  approach  as  typified  by  the  work  of 
Cajrrpbell  and  Stanley^  or  the  context^  input,  process  and  production 
(CIPP)  model  as  presented  by  Daniel  L.  Stuff leb earn. 

Specific  objectives: 

At  the  end  of  the  institute  th^  participants  will  demonstrate 
their  abilities  to  perform  the  follcwiyig  tasks: 

a)  Identify  the  classification  scheme  of  the  CIPP  evaluation  model. 

b)  Construct  a  research  study  using  either  context  evaluation^ 
input  evatuation^  process  evaluation  or  product  evaluxxtion. 

c)  Describe  the  difference  between  an  experimental' 'ayzd  a  .  . 
quasi-experimental  design  for  educational  research. 


The  fourth  major  theme  will  be  the  study  of  the  problems  associated 
with  the  development^  dissejninationj,  and  adoption  process  in  the  area 
of  educational  evaluation. 

Specific  objectives: 

At  the  end  of  the  in^^titute  the  participants  will  demonstrate  their 
ability  to  perform  tks -following  tasks: 

a)  Identify  the  problems  associated  with  designing  a  good  , 
research  evaluation. 

b)  Construct  a  research  study  which  reflects  understanding 
of  research  desigri  arid  measuranjnt  theory, 

c)  Distinguish  between  interpretations  which  are  in  fact 
justified  from  the . data  and  those  which  are  not  justified 
from  the  data* 

d)  Demonstrate  through  the  use  of  the  systems  approach  the 
problems  associated  with  developing:,  dissem.lnating^  and 
adopting  the  results  of  educational  research  within  an 
educational  environment. 


The  fifth  major  theme  will  be  the  study  of  cormunications  techniques 
applicable  to  proper  Vfrrplementation  of  the  decision-making  process 
at  various  levels  of  the  educational  system. 

Specific  objectives: 

At  the  end  of  the  institute  the  participants  will  demonstrate 
ikHr  ability  to  perform  the  following  tasks : 


a)  Interpret  the  research  findings  of  several^  journal  articles. 

b)  Demonstrate  the  competency  to  apply  the  findings  of  a 
research  study  to  one^s  local  scJiool  situation. 

c)  Describe  the  problems  associated  with  dissemination  and 
adoption  of  the  general  kind  of  educational  research 
fifidings  to  a  local  school  system. 

6.^       The  sixth  major  theme  Dill  be  the  use  of  modern,  data  processing 
equipment  to  facilitate  the  encoding  and  utilization  of  research 
data. 

Specific  objectives: 

At  the  end  of  the  institute  the  participants  will  demonstrate  their 
ability  to  perform  the  following  tasks: 

a)  Construct  a  simple  computer  program. 

b)  Identify  the  problems  associated  with  designing  an  optically 
scannable  docwnent. 

e)    Demonstrate  the  ability  to  operate  a  remote  terminal. 

d)  Name  and  describe  the  use  of  a  variety  of  modern  data 
processing  equipment  aiid  its  utilization  in  educational  research. 

?.  The  seventh  major  theme  will  be  the  reading  of  current  educational 
literature  relevant  to  the  vp.search  .project  with  which  the  trainee 
is  involved. 

Specific  Objectives: 

:  At  the  end  of  the  institute  th?  -participants  will  demonstrate  their 
ability  to  perform  the  followir^g  tasks: 

a)  To  identify  the  major  re fe^-r^en^e  sources      ^  educational 
research  literature. 

b)  Construct  a  bibliography  in  the  area  of  the  trainees^  interest.  . 

c)  Distinguish  the  major  components  of  a  piece  of  well  written 
educational  research. 
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^Although  this  objective  was  stated  in  the  original  proposal  as  a  requirement, 
for  the  instructional  program it  was  subsequently  made  optional  to  the 


GEOGRAPHICAL  REPRESENTATION  OF  PARTICIPANTS 


The  geographical  representation  o£  the  participants  in  this 
institute  was  as  follows: 


National 
1970 


Connecticut  1 

Florida  2 

Georgia  2 

Hawaii  1 

Iowa  2 

Louisiana  2 

Massachusetts  3 

Michigan  1 

Mississippi  2 

Montana  2 

New  Hampshire  1 

North  Carolian  1. 

Rhode  Island  1 

Siapan,.  Mariana  Islands  1 

South  Carolina  1 

Tennessee  3 

Texas  2 

Virginia  1 


29 

As  can  be  seen  from  Table  A  presented  above  the  institute  partici- 
pants  were  drawn  from  a  wide  and  diverse  geographical  area.    It  should  be 
noted  here,  that  the  total  number  of  trainees  was  twenty  nine,  although  the 
proposal  provided  for  the  funding  of  thirty-two.    Thirty  participant.^  were 
actually  accepted  for  the  institute,  however ^  one  had  to  ''drop  out'*  soon 
after  for  personal  (not  instituted  connected)  problems.    It  should  also  be 
noted,  that  in  the  originaT  proposal  under  the  heading  Number  and  Selection 
of  Participants  the  following  statement  was  made:     ''There  shall  be  no  conaezm 
for  geographical  preference  among  tioenty  (20)  of  the  participants .  They 
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wilt  he  acoepted  from  any  geographical  area  in  the  United  States  or  its 
territories.    Ten  of  the  participants  however^  shall'  he  black  educators 
from  the  southern  states*    Knoxville  College  in  Knoxoille^  Tennessee  has 
agreed  to  act  as  a  resource  for  helpuig  to  select  these  southern  educators.* 

In  retrospect,  without  the  assistance  of  Dr,  Sarah  Curwood  of 
the  University  of  New  Hampshire  and  Dr.  Ralph  llartin  of  Knoxville  College 
recruitment  and  selection  of  southern  black  educators  would  have  been 
virtually  impossible.    As  Dr.  Curwood  points  out  in  her  report,  _^^Advanced 
degrees  for  Southern  htacks  have  traditionally  heen  earned  hy  hurdling 
many  ohstacles.    Few  black  institutions  confered  master's  degrees:  fewer 
yet.  the  doctorate. '    Nevertheless >  through  persistent  efforts  on  the 
part  of  Dr.  Curwood  and  Dr.  'Martin>  eleven  southern  black  educators  were 
selected  as  participants. 

Additionally,  Dr.  Curwood  assumed  the  major  responsibility 
for  a  one  week>  pre-institute  orientation  in  order  to  provide  the  black 
educators  an  opportunity  to  adjust  to  the  different  cultural  pMterns  of 
New  England  as  opposed  ti  the  southern  states.    A  copy  of  Dr,  Curwood 's 
report  describing  the  events  and  activities  for  that  week  is  included 
in  Appendix  A  o£  this  report* 


ASSESSMENT  OF  IiNSTRUCTIONAL  PROGRAfl 

The  institute  was  conducted  daily  from  8:30  A*M.  to  4:00 
P.M.  for  six  weeks.    The  Department  of  Education  at  UNH 
offered  six  (6)  graduate  credits  for  two  courses  upon  the 
successful  completion  of  the  institute  program.  These 
two  courses  are  listed  below: 

Education  881,   (4  credits).  Methods  and  Techniques 
of  Educational  Research,    This  course  is  a  critical 
study  of  the  principal  methods  employed  in  the  in- 
vestigation of  educational  problems  and  an  evaluation 
of  the  procedures  and  standards  used  in  reporting 
the  findings;  designed  as  an  advanced  course  for 
candidates  for  graduate  degrees. 

Education  895,   (2  credits),  Research  Problems  in 
Education,  is  concerned  with  the  individual  inves- 
tigation of  a  problem  in  the  area  of  educational 
research.    This  course  was  used  first  to  expose  the 
trainees  to  a  variety  of  educational  research.  In 
the  latter  portion,  each  trainee  was  expected  to 
develop  a  research  proposal  which  will  have  practical  , 
importance  and  relevance  to  his  home  district,  or  to' 
evaluate  a  project  presently  being  studied. 

In  an  attempt  to  determine  the  adequacy  or  inadequacy  of  the 
instructional  objectives,  the  following  assessment  tasks  were  undertaken 
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A  pre-  and  post-test  was  constructed  which  consisted  of  160  multiple  choice 
items.    These  questions  were  in  general  drawn  from  a  test  designed 
by  Gene  V.  Glass  entitled,  "Mastery  Test  Items  For  Courses  in  Educational 
Research  Methods"  a  copy  of  this  pre-  and  post-test  will  be  found  in  Appendix. 
A  of  this  report.    This  test  had  been  used  in  the  1969  summer  institute  and 
was  found  to  be  very  reliable  and  an  adequate  measure  of  the  learning  that 
took  place  in  that  sununer's  institute. 

A  complete  summary  of  statistical  tables*  based  on  both  the  pre- 
and  post-test  results  are  presented,  beginning  on  page  11 .    They  consist  of 
the  following: 


Tcijle  I 
Table  II 


Table  III 


The  raw  scores  on  the  pre-test,  post-test  and  the  gain  score 
and  percent  of  gain  for  each  participant. 

Provides  the  mean  and  standard  deviation  as  well  as  the 
range  of  both  the  pre-  and  post-test.    On  the  pre-test  the 
mean  was  73.96  with  a  standard  deviation  of  29.48,  The 
range  for  the  pre-test  was  from  a  low  of  23  to  a  high  of 
115,  vjith  a  range  of  92  score  points.    For  the  post-test 
the  mean  was  85,29  with  a  standard  deviation  of  22,97. 
The  low  score  was  45  and  the  high  score  was  128,  with  a 
raw  score  range  of  83  score  points.    This  indicates  that 
over  the  summer  there  was  an  11 , 33  raw  score  point  gain  from 
pre  to  post-test,  which  is  significant  at  beyond  the  ,01 
level. 

Presents  the  Pearson  Product  Moment  Correlation  between 
the  pre-test  and  the  post-test.    The  correlation  is  .84. 
This  correlation  coefficient  of  ,84  indicates  a  relatively 
high  positive  correlation  over  the  six  weeic  period.    The  ' 
increase  in  scores  seems  to  occur  evenly  throughout  the 
sample  meaning  that  people  who  did  well  on  the  pre-test 
also  did  well  on  the  post-test.    This  correlation  coefficient 
is  significant  from  chance  at  greater  than  the  .001  level. 
The  square  of  this  correlation  coefficient  is  .71  indicating  a 
much  better  than  chance  probability  of  predicting  one  score 
from  the  other. 


*It  should  be  noted  that  all  of  these  statistics  are  based  on  an  N  of  24  / 
participants .    There  were  in  fact  thirty  participants  in  the  institute  but 
;two  of  them  were  used  by  the  outside  evaluator  as  participant  observers  and 
cofr'^'^®^®^"^®  ^^^^^^^^"^  to  take  the  test .    One  "dropped  out"  for  personal 

bryLsasons,  and  three  others  arrived  several  days  after  the  institute  started. 


Table  IV  Presents  the  calculations  for  the  Spearman  Rank  Correlation 

Coefficient  and  indicates  both  the  pre-test  rank  and  the 
post-test  rank  as  well  as  the  relative  difference.  The 
correlation  coefficient  hero  is  .86  which  is  significant  at 
beyond  the  .001  level. 


s  are 
e 

88, 


Table  V  In  Table  V  and  VI,  split  half  reliability  coefficient 

l^hT.  I/V  con^P^ted  for  the  pre-  and  post-test.    For  the  pre-test  the 

Table  VI  reliability  coefficient  is  .96,  for  the  post-test  it  is  .8 

Careful  examination  of  these  statistics  would  seem  to  indicate  that 
the  institute  was  "reasonably  successful"  in  fulfilling  its  instructional 
objectives  as  measured  by  this  particular  test.    But  permit  us  to  digress  for 
the  moment.     It  was  noted  earlier  in  this  report  that  one  specific  criteria 
for  selection  of  ten  of  the  participants  was  that  chey  be  southern  black  educa- 
tors.   Actordingly,  one  interesting  dimension  of  the  preceeding  assessment  data 
is  worth  noting.    In  general,  on  both  the  pre-  and  post-test  the  performance 
level  of  the  southern  black  participants  was  relatively  less  than  the  average 
white  participant.    On  the  other  hand,  of  the  twenty-four  (24)  scores  reported, 
ten  black  participants  achieved  the  highest  percent  gain,  indicating  that 
they  benefited  very  markedly  from  the  instruction  offered  to  them. 

In  addition  to  the  so-called  Glass  pre-  and  post-tests,  the  partici- 
pants were  administered  an  Attitudinal  Scale  developed  by  C. I. R.C.E.*  As 
judged  by  its  authors,  the  instrument  distinguishes  changes  in  attitude  relative 
to  the  broad  subject  area  of  evaluation  over  a  period  of  time.    There  were  five 
general  areas  of  attitude  change  that  were  measured  as  well  as  an  overall 
composite  score  called  '^Confidence  in  EvaluatJon."    The  five  sub-scores 
concern  attitudes  about  research,  service,  teaching,  objectives  and  judgment. 
The  results  of  the  pre-  and  post-attitudinal  scale  are  offered  in  Table  VII; 


Er|C^  ^PP^^^ix  B  -  CIRCE  Attitude  Scale  No.  1.4  and  i;4b 


-10- 

Table  VII  indicates  that  on  five  of  the  six  measures  there  was  a  positive 
attitudinal  shift  over  the  six-week  period.    Only  in  s'nb-measure  four  entitled 
oMectives  was  there  a  negative  attitudinal  shift  in  the  rating  scale  over  the 
period  of  six  weeks. 

Thus,  it  would  seeii.  reascnable  to  conclude  from  the  results  of  this 
attitude  scale  that,  in  general,  participant  attitude.^'  about  evaluation  became 
more  positive  as  the  six  ;veek  program  di'^v'  to  a  close. 

It  is  also  interesting  to  lock  at  Table  VIII*  which  presents  the 
CIRCE  Attitude  Scale  broken  down  by  black  vnd  white  participants .  Observe, 
that  again  the  black  participants  seemed  to  have  shifted  their  attitudes  about 
educational  evaluation  markedly  in  a  positive  direction.    This  again  reflecting 
that  the  instructional  program  responded  adequately  to  the  expectations  held 
by  the  black  participants. 

Based  upon  the  instructional  program  and  other  related  experiences 
gpin^^d  during  the  six  week  institute,  <::li  participants  were  expected  to  develop 
a  research  and/or  evaluation  proposal  which  had  practical  importance  and  relevance 
"back  home  J* 

AbFtracts  of  these  proposaJs  are  included  in  Appendix  A. 

The  material  presented  in  thi:;  section  of  the  report,  seems  to  support 
the  conclusion  that  in  terms  of  cognitive  knowledge  and  attitudinal  changes 
relative  to  educational  research  and  educational  evaluation,  the  insl&^ate  was 
^'reasonably  successful*'  in  .attaining  its  instructional  objectives. 


*The  same  information  is  also  presented  in  graphic  format  --  Ref:  Table  VIII  - 
Q  Figure  I. 
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TABLE  J 

Pre-  and  Post-test  Raw  Scores,  Gain  Scores  and  Percent  o£  Gain 

PARTICIPANT  PER  CENT 

mmK  "-L  V  GAIN  GAIN 


01 

39 

0 

15 , 4 

02 

61 

11 . 0 

03 

114 

04 

25 

71 

AS 

2(38;7 

05 

81 

r  O 

-5 

-3,  7 

06 

29 

35 

124, 1 

07 

7S 

-9 

-12,0 

08 

71 

30, 9 

09 

48 

O  1 

6.3 

10 

46 

80 

77  Q 

11 

91 

97 

6 

6.6 

12 

:  108 

128 

20 

18.5 

13 

55 

72 

17 

30,9 

14 

110 

115 

5 

4.5 

15 

70 

75 

5 

7.1 

16 

112 

125 

13 

11.6 

17 

97 

93 

-4 

-4.1 

18 

84 

86 

2 

2.4 

19 

66 

92 

26 

39.4 

20 

104. 

95 

-9 

-8.7 

21 

113 

122 

9 

7.9 

22 

72 

76 

4 

5.6. 

23 

91 

83 

-8 

-8.8 

24 

25 

64 

39 

156.0 

ERIC 
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TABLE  II_ 
Pre -Test 

Meim  ®f  :X  =    7$  .  96 

Standard  Deviation  of  X  =  29,48 

Range:    23  -  115  (92) 


Post -Test 


Mean  of  y  =    ^  J^^ 

Staftd,3lfd  Deviation  of  t  =    22. §7 
Range:    45  -  128  (83) 
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TABLE  III 

Pearson  Product  Moment  Correlation  Coefficient 
(correlation  coefficient  -  .84) 


X 

Y 

• 

39 

45 

50 

61 

115 

114 

23 

71 

81 

78 

29 

65 

75 

(66 

71 

93 

48 

51 

46 

*T  \J 

80 

91 

97 

108 

128 

55 

72 

110 

115 

70 

75 

112 

125 

97 

93 

84 

86 

66 

92 

104 

95 

113 

122 

72 

76 

91 

83 

25 

64 

-14- 


TABLE  IV 

Spearman  Rank  Correlation  Coefficient. 


(correlation  coefficient  -  .86) 


Participant 

NuiTiber 

X 

Y 

D 

* 

03 

1 

5 

-4.0 

21 

2 

3 

-1.0 

16 

3 

2 

1.0 

14 

4 

4 

0.0 

12 

5 

1 

4.0 

20 

6 

7 

-1.0 

17  ■ 

/ 

8.5 

-1.5 

11 

8.5 

6 

2.5 

23 

8.5 

12  ■ 

-3.5 

18 

10 

11  ■ 

-1.0 

05 

11 

14 

-3.0 

22 

13 

15 

-2.0 

08 

14 

8.5 

5.5 

15 

15 

16 

-1.0 

19 

16 

10 

6.0 

13 

17 

17 

0.0 

02 

18 

22 

-4.0,, 

09 

19 

23 

-4.0 

10 

20 

13 

7.0 

01 

21 

24 

-3.0 

06 

22 

20 

2.0 

24 

23 

21 

2.0 

04 

24 

18 

6.0 
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TABLE  V 

Reliability  Coefficient  --  Pre-Test 
RAW  SCORE  EVEN  RAW  SCORE  ODD 


22 

17 

22 

28 

53 

62 

12 

11 

42 

39 

13 

16 

42 

33 

36 

35 

23 

,  25 

24 

22 

43 

48 

55' 

53 

26 

29 

53 

57 

27 

33 

59 

63 

47 

50 

39 

45 

33 

33 

51 

53 

58 

55 

35 

37 

40 

41 

9 

16 

RELIABILITY  COEFFICIENT  .9614 
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TABLE  n  , 
Reliability  Coefficient  Post-Test 
RAW  SCORE  EVEN  RAW  SCORE  ODD 


If 

42 

3o 

38 

Ob 

5/ 

bi 

,44 

A  A 

4U 

/in 

46 

bU 

b3 

62 

40 

r  c 

:bb 

60 

35 

37 

64 

64 

49   \  : 

48 

36 

44 

26 

52 

32 

•■•:.^'34 

32 

■  03 

39 

39 

38-.  . 

■  33 

59 

U 

v..  ■,27 

22 

■■.  22 

RELIABILITY  COEFFICIENT  .8852 
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'ABLE  VII 


C.I.R.C.E.  Pre-  and  Post  lie asure  of  Attitudinal  Change 


ALL  PARTICIPANTS 


.  :  SCALE 

1^ 

PRE 

POST 

Research 

,  ■  27' 

4.0 

4.9 

Service 

27 

5.5 

6.3 

Teaching 

27 

5.0 

6.6 

Objectives 

■  27 

5:6 

5.2 

Judgment 

27 

'  6.5 

7.2 

Overall  Confidence  in  Evaluation 

.  27 

8.1 

9.6 

TABLE  VIII 


C.I.R.C.E.  Pre-Post  Measure  of  Attitudinal  Change 


BLACK  AND  WHITE  PARTICIPANTS 


ERIC 


BLACKS 

PRE 

POST 

Research  - 

9- 

4,7 

10- 

6.5 

Service 

9- 

4.7 

10- 

5.5 

Teaching 

4.8 

10- 

7.0 

Objectives 

9- 

4.9 

:  '  10- 

5.8 

Judgment 

9- 

5.6 

■  :  10- 

7.0 

Overal?.  Confidence  in 

Evaluation 

9-  ; 

7.0 

10- 

9.3 

WHITES 

PRE 

POST 

Research 

:  18- 

3.6 

17- 

3.9 

Service 

;  18- 

5.9  ■ 

17- 

6.8 

Teaching 

18- 

5.1 

17- 

6.4 

Objectives 

18- 

5,9 

17- 

4.9 

Judgment 

18- 

6.9 

17- 

7.3 

Overall  Confidence  in 

Evaluation 

18- 

8.7 

17- 

9.7 

rt 

i  : 

II  •  • 

n 


TABLE  VIII- -  FIGUEE  I  : 
rr  UinRVC.E.  :PRE-P0ST  MEASURE  OF  ATTITUDINAL  CHANGE* 
•BLACK-AND-WHITE  PARTICIPANTS 
■     '(Based  on  Average  Responses)   


WHITES-Post 


}  I 

i  ; 
I         J  ■ 
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EVALUATION  OF  THEi  1970  SUMMER  RESEARCH 
TRAINING  INSTITUTE 


''Planning  for  Educational  Decision-Making 
in  the  70' s^' 

July  6  -  August  14,  1970 

.  ■  ■  ■    '  ' ■  ■ 

SECTION  II 
. PROCESS  EVALUATION 

-    ,       "   ■  ■ ,  ■        ,  ■  r 
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PROCESS  EVALUATION* 

In  carrying  out  the  evaluative  functions  of  the  1970  HEIV  Institute 
at  UNH,  GIRCiB  employed  several  information-gathering  techniques.    For  the 
process  evaluation,  they  relied  on  two  major  sources  of  information:  several 
one -to -three -day  on-site  visits  by  the  evaluators  to  the  Institute,  during 
which  they  observed,  listened,  and  probed;  and  a  Weekly  Participant  Opinion- 
naire,  by  means  of  which  they  obtained  systematic  feedback  from  the 
participants  about  their  reactions  to  each  week's  activities,     (A  copy  of 
this  Opinionnaire  and  all  other  instruments  used  in  their  evaluation  will  be 
found  in  Appendix  B  to  this  report.)    An  early  expectation  was  that  the 
participant  observers  would  provide  the  evaluators  with  a  third  kind  of 
information  through  a  series  of  regular  reports. 

For  a  summative  evaluation,  they  made  use  of  some  of  the  ^'process'' 
information  obtained  during  the  course  of  the  Institute  but  incorporated  other 
^data  gathered  by  means  of  (1)  an  Orientation  to  Evaluation  scale  given  at  the 
beginning  and  again  at  the  end  of  the  Institute,  (2)  ^*self -interviews"  from 
the  instructional  staff  (consultants),  and  (3)  a  Summary  Opinionnaire.  Copies 
of  these  instruments  are  also  included  in  Appendix  B.     (On  the  Summary 
Opinionnaire,  frequencies  of  responses  to  the  various  alternatives  have  been 
provided. ) 


*This  section  is  being  presented  as  information  relative  to  a  process  evaluation 
of  the : 1970  HEW  Institute.    The  process  evaluation  was  contracted  to  the  Center 
for  Instructional  Research  and  Curriculum  Evaluation,  College  of  Education, 
University  of  Illinois,  Urbana,  Illinois      61801.    It  should  be  noted,  that  the 
italicized  section  which  follows  is  a  condensed  version  of  the  CIRCE  Final 
Evaluation  Report . 

o  ■  .  ■  •   •  ■ 
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ADMINISTRATIVE.  STAFF 


In  any  educational  endeavor y  some  things  are  done  rights  some  wrong. 
The  former  teiid  to  get  overlooked:,  the  latter  overemphasised.    Although  we  have 
tried  in  this  report  not  to  overlook  the  things  that  were  done  right  and  not 
to  overemphasize  the  things  that  were  done  wrong ^  it  is  unlikely  that  we  have 
heen  ccmpZetely  successful. 

The  Institute  staff  —  Elwell^  Barnes^  Olivier.^  and  others  from 
the  University  of  New  Hampshire  ~  did  an  excellent  job  of  recruiting  top- 
quality  participants  in  a  very  short  time.     The  ^'General  Information  Handbook/* 
designed  by  Elaine  Gardnerj  Institute  secretary  and  technical  staff  assistant^ 
and  issued  to  all  participants  and  consultants:,  was  'also  a  very  thoughtful 
and  useful  dociment  insofar  as  it  provided  genera^^  to  the 

poT'ticipants  and  instructional  consultants  about  the  Institute  and  the  local 
a:rea\  ■  .  ■    ■ ; 

Throughout  the  Instifute  these  administrative  staff  members  showed 
themselves  to  be  energetic^  patient^  and  attentive  to  and  concerned  about 
the  needs  and  desires  of  the  Institute  participants.    In  support  of  this 
statement y  we  have  participant  opinions  on  the  weekly  opinionnaire  to  the 
effect  that  the  staff  was  available  to  talk  to  whenever  necessary^  pco^tiaularly 
from  the  third  week  on.     (During  weeks  one  and  two  of  the  Institute  four 
participants  were  less  than  satisfied  with  staff  availability  but  from  that 
ti^e  on  everyone  agreed  that  the  staff  was  indeed  readily  available. ) 
Similarly^  participants  said  t     staff  was  paying  enough  attention  to  the 
important  problems  that  arose  although  a  couple  of  participants  were  never 
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satisfied  in  this  respect.    Participants  also  felt  that  the  staff  had  taken 
appropriate  actions  to  resolve  problems  as  they  arose.    Finally^  on  the  Swma:ry 
Opinionnaire^  21  participants  rated  the  staff  as  ''Good'^  or  ''Excellent "  while  ' 
three  rated  th3m  as  ''Poor^'  or  ''Bad.  ^ 

Institute  staff  did  an  excellent  job  of  making  themselves  available  to  listen 
to       and  at  least  gave  the  appearance  of  acting  to  correct       the  problems 
of  the  participants  as  they  arose  during  the  coicpse  of  the  Institute. 

An  important ■ component  of  the  dutieo  of  the  staff  of  a  summer 
institute,  however^  is  to  facilitate  the  smooth  flow  of  the  instructional 
.   component  of  that  institute.     THIS  IS  PABTICULARLY  IMPORTANT  IN  AN  INSTITUTE 
ORGANIZED  LIKE  THE  ONE  IN  NEW  MI^SHIRE  WHERE  PARTICIF^^  TOTAL  OF 

SIX  WEEKS  ~  ARE  EXPOSED  TO  SEV  "EXPERTS/''  EACH  OF  WHOM  HAS 

/  A  UNIQUE  TOPIC  TO  PRESENT  AND  A  UNIQUE  METHOD  FOR  PRESENTING  THAT  CONTENT .  If 
the  presentations  of  guest  consulta7zts  are  to  be  "relevant"  to  the  theme  of 
the  institute  and  to  the  needs  of  the  participants y  if  the  transition  from 
^ :^oonsiAl^  (topic  to  topic)  is  to  be  accomplished  smoothly ,  and 

if  time -consuming  and  threatening  confrontations  of  participants  and  consultants 
are  to  be  mini-^ized^,  it  is  imperative  that  the  permanent  administrative  staff 
of  the  institute  take  strong  positive  action,     l^t  kinds  of  actions,  might 
serve  to  facilitate  the  flow  and  transition  of  instruction?    Firsts  it  would 
,     seem  necessary  that  participants  be  given  some  advance  information  about  each 
consultant.    This  might  include  an  overview  of  the  content  of  h,is  presentation 
(perhaps  consultants— -  as  a  part  of  their  responsibilities  to  the  institute 
should  prepare  such  overviews  themselves)  several  days  before  the  consultant 
is  scheduled  to  "perform.  "    In  addition^  it  might}  help  the  participants  if 

ERIC";::.,;::::  ■■. 
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they  were  to  receive  a  brief  vi  'si  of  each  consultant.    Activities  like  these 
last  tioo  would  give  the  pca;^ticipants  some  idea  about  what  the  consvltants 
were  going  to  pvesentj  thereby  alleviating  some  of  the  anxieties.  (That 
such  anxieties  among  participants  did  in  fact  exist  in  the  New  Hampshire 
Institute  is  reflected  in  reports  from  the  participant  observers  during  the 
early  parts  of  the  Institute  and  in  some  of  the  oorments  made  to  the  evaluators 
during  site  visits. ) 

Equally  ac  impovtairb  as  prepa:oing  the  institute  participants  for  the 
consultants  is  the  task  of  preparing  the  consultants  for  the  participants. 
:  That  isj^  consultants  should  he  provided  - —  in  advance  —  with  information  about 
the  institute  participants:    their  backgrounds^  job  descriptions^  purposes  in 
attending  the  institute^  and  any  information  available  about  what  to  expect 
by  Way  of  group  dynamics  during  the  institute.     (This  latter  item  will  be  most 
useful  to  the  second  and  subsequent  consultants , )  > 

had  been  available  during  the  New  Hampshire  Institute .it  is  likely  that  many 
of  the  consult  ant -^participant  conflicts  could  have  been  avoided  m   ■  Tt  is  on 
this  issue  tJiat  the  administrative  staff  of  the  New  Hampshire  Institute  should 
be  taken  to  task.    True:,  the  Institute  staff  was  handicapped  by  a  lack  of  time 
be'tcy2:^n  date  of  funding  and  the  start  of  the  Institute.     True^  the  Institute 
:  director  stepped  in  at  the  last  minute  to  replace  the  person  who  had  planned  ■ 
the  Institute with  a  very  different  philosophy  and  a  very  different  adminis-^ 
trative  style.    Nevertheless:,  [t^^  task  of  preparing  participants 

for  the  Qonsu Itants  and  consultants  for  the  participants  cou Id  have  been 
better  handled.    The  lateness  of  funding  undoubtedly  was  a  factor  here. 


In  spite  of  the  above  ocrments^  this  report  would  not  he  complete  or  ' 
fair  without  mention  of  one  outstanding  counter -exQjnple.  :  After  ti)o  and  one- 
half  loeeks  of  thia  Institute,  it  was  apparent  that  consultants  could  not  come 
in  and    do  their  thing'*  for  their  allot eS.  time  without  careful  consideration 
of -the  needs  and.  desires  of  the  participants.     On  the  basis  of  the  experiences 
of  Cook  and  Walbesserj  the  Institute  director  made  a  personal  visit  to  the 
next  consultant^  Daniel  Stufflebeam^  several  days  in  advance  of  his  scheduled 
participation  in  order  to  describe  to  him  the  kinds  of  difficulties  the  first 
two  consultants  had  experienced.     On  the  basis  of  that  advance  notification^ 
Stuff lebeam  made  some  modifications  in  his  proposed  method  of  operation  for 
his  two-week  segment. 

^    •  .  STAFF:.    PARTICIPANT^  OBSERVERS 

Two  participant  observers^  one  black  and  one  white were  recruited 
foi*  the  New  Hampshire  I?istitute:     Their  primary  allegiance  was'  to  the  Institute 
director  although  it  was  anticipated  that  their  observations  would  also  he  of 
considerable  use  to  the  evaluators.     That  they  fulfilled  their  primary  role  ~ 
that  of  monitoring  and  reporting  to  the  Institute  director  the  feelings  of  the 
pax'ticipants  about  the  Institute — can  be  attested  to  by  the  administrative 
staff.    Thai  they  were  of  less  use  than  anticipated  in  their  secondary  role — 
that  of  providing  useful  informat^ion  to  the  evaluation  team  during  the 
Institute  ~~  can  be  attested  to  by  the  evaluators .    This  lack  of  utility  to 
the  evaluators  of  the  participant  observers  raises  some  important  questions 
about  how  evaluators  can  obtain  the  most  benefit  —  or  any  benefit  —  from 
the  presence  of  such  observers . :  These  questions  will  be  considered  later  in 
this  report. 
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'      .        An  important  factor  to  consider  in  the  use  of  participant  observers 
is.  the  careful  specification  of  their  role  in  an  institute.    Are  they  to  be 
.  ....  ■p2!i/nari2t/ Jporti  It  is  likely  that  this 

question  will  ajiswer  itself ^  given  the  context  in  which  i*^  participant 
observer  is  functioning  and  his  personality:     It  is  also  important  to  con- 
sider the  unwanted  side  effects  of  the  presence  of  participant  observers.  In 
the  New  Hampshire  Institute  thej'^e  was  some  early  resentment  on  the  part  of  some 
participants  to  the  presence  of  the  observers.^  especiall'j  in  the  residential 
setting.    As  one  participant  remarked:,    It  gets  annoying  to  have  someone 
looking  over  your  shoulder  while  you're  eati?ig  dinner noting  who  you  are 
talking  to  and  what  you  are  saying./^    Yet from  the  point  of  view  of  the 
Institute  staff ^  the  information  gleaned  by  a  participant  observer  in  that 
|-  setting  was  and  is  an  invaluable  asset  to  on-going-  operations  and  administra-  . 
tive  decision  making.    Related  to  this  last  pointy  on  the  Summary  Qpinidnnaire 
fifteen  participants  rated  the  participant  observers  as  good^ 
uncertain^,  and  the  other  five  gave  decidedly  negative  responses.    In  the: 

  -^^  generally,  positive  reactions  to  the  Institute^  these  responses- 

represent  what  might  be  significant  dissatisfaction.     Unfortunately^  perhaps, 
.   it  is  impossible  to  differentiate  between  the  two  obsei^ers  to  determine 
whether  either  or  both  represented  a  source  of  dissatisfaction. 

CONSULTANTS  .    .  . 

The  prospectus  for  the  New  Hampshire  Institute  indicated  five  major 
instructional  components:    Week  One  —  Desmond  Cook  on  project  management  models 
;   and  techniques;  Weeks  Two  and  Three  —  Henry  Walbesser  and  staff  on  behavioral 
,    objectives^  design  of  assessment  tasks j  and  construction  of  learning  hierarchies:; 


Weeks  Four  and  Five  —  Daniel  Styy.fftebeojriy  Egon  Guba,  and  Robert  Hammond  on  ... 
evaluation  models  and  techniques:,  s^Bminars  on  problematic  simulations;  and  Week 
Six  "  Williojn  Asher  and  Daniel  Heisey  on  px^oble^ns  associated  with  developing^ 
disseminating ,  and  adopting  educational  innovations.     This  lineup  was  described 
-in  the  planning  group  meeting  prior  to  the  Institute  (May  24)  as  a  '^parade  of 
stars.  " 

It  is  thft  irrrpression  of  the  evaluators  that  many  of  the  problems  • 
arising  from  consultant-^participant  or  consulta:>%t-staff  encounters  in  the 
New  Hampshire  Institute  resulted' from  the  fact  that  the  two  group  combinations 
held  very  different  sets  of  expectations  as  to  what  the  Institute  was  all 
about »     The.  consultants^  who  had  had  much  experience  with  (traditional)  summer 
institutes^  came  with  the  expectation  that  they  were  going  to  spend  one  or 
two  weeks  ^^doing  their  thing.  ''     The  participa?its^  many  of  whom  had  not 
been  initiated  into  the  fraternity  of  summer-institute  attenders^  came  look-^ 
ing  for  material  which  would 'be  directly  relevant  to  them  as  they  performed 
in  their  prof e^iiionaZ  roles  '^^aclc  home.-    Be<so:us,e  the  participants ^  as  a 
group  J,  were  extremely  heterogeneous  with  respect  to  whc:t  was  "''relevant  to 
them'^j  their  individual  expectations  were  impossible  of  fulfillment  in  the 
context  of  th(>  Institute,     Because  the  first  two  consulvants  were, not 
initially  aWare  of  (2)  the  diverse  naturae  of  the  group  and  (2)  the  expecta-- 
tions  of  the  participants^  they  Ivad  no  chance  even  to  attempt  to  meet  those 
expectations* 

Given  that  it  is  impossible  to  control  the  personal  style  of 
consultants it  is  imperative  that  f he  administrative  staff  of  an  Institute 
such  as  this  one  work  to  insure  that  ocnfrontations  are  avoided  or  consultants 
are  prepared  for  them. 
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How  did  the  'participants  themselves  react  to  these  events?  Data 
relevant  tc  this  question  can  he  gleaned  from  the  Weekly  Participant  Opinion- 
naires.    Firsts  did  participants  enjoy  the  instructional  staff  (consultants) 
each  week? 


Yes 

Yes  but. . . 

No 

Week  One  (Cook) 

16 

■  9 

2 

Week  Two  (Walbesser) 

'25 

2 

7 

Week  Tlrree  (Walbesser) 

■  19 

3 

4 

Week  Four  (StuffZebecon) 

25 

2 

Week  Five  (Stuffleheamj, 
Guba  and  Hammond) 

27 

Week  Six  (Asher^  Heisey) 

NO 

DATA  COLLECTED 

Clearly  from  this  point  of  view^  Week  One  was  not  badj  Week  Two  not  too 
goodj,  Week  Three  pretty  good^  and  Weeks  Four]  and  Five  excellent. 

But  there  are  other  aspects  of  these  instructional  blocks  which 
mu^t  be  considered.     Whether  or  not  a  participant  enjoyed  the  consultant) s) 
for  a  particular  week^^  did  he  learn  alot?  Observe: 

Yes      Yes  but. . .No 
Week  One  17  '        ^  3 

Week  Two  12  3       -  :  6 

Week' Three  11  5      ,  8 

Week  Four  20  5 

Week  Five  2? 

As  anticipated^  there  is  a  good  bit  of  consistency  in  the  distribution  of 
responses  to  these  two  questions.  ' 
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It  Was  during  the  third  week  of  the  Institi-cte  that  the  director^ 
Elwellj,  met  with  StuffZebeam.    As  a  result  of  that  meeting.  Stuffleheam 
made  an  effort  to  adapt  to  the  needs  and  desires  of  the  participants.  Among 
his  more  successful  tactics  were  (1)  requesting  t^iat  the  participants  tell 
him  what  kinds  of  problems  they  were  concerned  with  and  would  like  help  on; 
(2)  setting  up  small -group  sessions  focusing  on  common  problems;  and  (3) 
appointing^  after  consulting  with  the  Institute  staff  on  the  Sunday  evening 
preceeding  his  first  instructional  day an  '^advisory  group"  of  participants 
to  assist  in  planning  the  Week^s  activities^     One  of  the  evaluators  (Wardrop) 
was  present  during  StuffZebeam 's  initial  encounters  with  the  participants. 
It  was  that  evaluator  ^ s  opinion^  that  once  StuffZebeam^had  ^taken  those  three 
initial  actions s  he  could  proceed  in  whatever  manner  he  chose  with  the  full 
compliance  of  the  participants. 

PAETICIPANTS     .  _ 

A  leitmotiv  throughout  th3  New  Harrtpshire%Institute  was  the  interplay 
of  the  diversity  of  backgrounds ;  abilities^  responsibiZi ties y  and  conc^^  of 
the  participants.    Such  heterogeniety  was  built  into  the  Institute  by  the 
criteria  for  selection  of  participants-  in  the  original:  proposal.    The  Institute 
was  to  serve  graduate  students LEASj^  and  personnel  in  ctate  and  federaZ 
agencies  and  was  to  incZude  specif icaZZy  ten  southern  bZacks.    As  a  consequence 
of  the  seZection  process y  the  Institute  wo'urid  up  29  taZented  (foT  the 

most  part) J,  concerned  (without  exception)^,  and  aggressive  (in  comparison  with 
other  groups  in  previous  institutes)  participants^    Such  a  mix  was  perhaps 
the  singZe  most  important  caus 

Institute.    1-6  created  probZems  for  the  consultants  from  the  very  beginning: 
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[One  of  the  Institute^ s  greatest  wec<kne^Qes  was]  the  heter- 
ogeniety  of  the  group  in  both  hackgroimd  experiences  and 
educational  levels.     (Consultant  A) 

[One  of  the  Institute' 8  greatest  weaknesses  Was]  the  diversity 
of  the  purposes  for  which  the  participants  were  attending  the 
Institute. .\  .  (Consultant  B) 

[One  of  the  Institute '^s  greatest  weaknesses  was]  hetero- 
geneous participants.     ( Consultant  C) 

[07ie  of  the  Institute's  greatest  weaknesses  was]  too  much 
variance  amoTig  the  partioipanto  in  both  motive  for  attending  \ 
and  academi  .  ability.     (Consultant  D) 

Based  on  experiences  in  this  Institute  (and  elsewhere) ^  we  feel 
impelled  to  raise  some  important  question.    about  the  iiature  of  the  selection 
process.    These  questions  will  he  considered  in  c  later"  section  of  this 
report. 

-     '  INSTRUCTIONAL  CONTENT 

Just  as  diversity  characterized  the  participants  in  the  New  Hampshire 
Institute^  so' the  cry  for  relevance  characterized  their  feelings  about  the 
ins  tructional  program  of  the  Instit  "Program  mdnagemehtj  ''  ^'behavioral 

oboeetives^     '^lear7ting  hierarchies^     ^'the  CIPP  eva lua  tion  mode  I ^  "  and  ^'compre- 
hensive achievement  monitoring^^  are  fashionable  among  educational  researchers 
and  theorists;  but  some  participants  in  the  New  Hampshire  Institute  could  not 
see  the  possibility  of  applying  these  techniques  and  procedures  to  the 
problems  with  which  they  weir e  themselves  concerned They  did  not—  many  of  ■ 
them^  at  least  —  want  their  time ^  ^ ^Wasted.      l^Uiether  this  was  the  feeling 
of  the  majority  of  the  participants  does  not  matter  for  it  was  the  feeling  ? 
of  at  least  a  very  vocal  group. 


This  ory  for  ''relevance"  created  some  unique  problems  because  what 
ms  relevant  for  any  half-dozen  participants  was  not  for  the  remaining  twnety- 
two.    Given  the  diversity  of  the  participants ^  satisfying  a  demand  for 
relevance  of  the  instructional  program  simply  was  not  possible.    The  only 
course  of  action  which  had  hope  of  succeeding  was  individualization  of 
instruction.    But  total  individualization  was  itself  not  feasible^  given  the 
limited  amount  of  time  and  energy  of  a^xy  one  of  the  consultants.    It  was 
most  nearly  approached  during  the  fifth  week  with  Stuff lebeoni:,  Guba^  and 
Hamnond  as  an  effective  instructional  team.    Inadequate  knowledge  — on  the 
part  of  both  staff  and  consultants       of  the  backgrounds  and  interests  of 
the  participants  was  another  limiting  factor.    The  participants  and  participant 
observers  were  themselves  aware  of  this  limitation^^  as  evidenced  by  their 
suggestions  for  '^pretestings'  the  participants  in  order  to  place  them  iyi 
small  groups  for  certain  parts  of  the  instructional  program.      ■     '  >  — 

Both  the  cry  for  relevance  and  the  desire  for  individualized  or 
small-group  instruction  Jiad  their  counterpoints ^  however.    In  the  former 
case  it  is  important  to  keep  in  mind  the  viewpoint  expressed  by  one  of  the 
participants  (addroceing  Stufflebeam}  :    *^I  came  here  to  learn: .  .  to  hear  and 
learn.    I  might  not  know  what  you  're  talking  dboutj,  I  might  not  understand 
you^  but  whenever  you^ re  talking,  I'll  be  sitting  there  U string.''  For 
people  like  this  one y  the  exposure  to  the    star s^l  xoas  important.  Relevance^ 
for  him,  was  not ^ a  primary  criterion.    In  the  latter  case  (individualized  or 
small-grouped  instruction)^  an  interesting  xmd^jpuzzling  phenomenon  occurred: 
Ifhen  Stufflebeam  in  Week  Four  strove  valiantly^ Jtp  divide  the  participants  ■ 
into .  small  groups ,  with  at  least  a  modicum  of  commonality  of  interest  and 
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conoerns  within  ea-^h  group,  he  met  wi-th  resistance.    He  found  that  ojzly  if 
he  personally  would  meet  with  every  small  group  would  many  of  tlie  participants, 
aooept  that  approach  to  instruction.    Everyone  —  or  nearly  everyone  —  insisted 
on  baing  where  Stuffleh&am  was  lest  they  "miss  something.      Does  this 
phenomenon  have  some  implications  for  the  kinds  of  selection  strategies  and 
criteria  which  ought  to  he  built  into  institutes? 

Questionnaire  responses  provide  further  information  about  the 
reactions  of  par tic.        s  to  the  content  of  the  instructional  program.  Perhaps 
the  most  important  question  in  this  realm  —  on  a  week-to-week  basis,  at 
least  —asked  if  the  week's  content  had  been  challenging  and  important.  The 
week-'by^Week  responses  Were  as  follows: 


Yes 

Yes  but. . . 

No 

Don't 

Week  One  (Cook) 

18 

•9 

Week  Two  (Walbesser) 

15 

6 

3 

1 

Week  Three  (Walbesser) 

14 

7 

6 

2 

Week  Four  (Stuff leheam) 

21 

Week  Five  (Stuff lebeam^ 
Cuba  and  Hammond) 

24 

1 

With  the  exception  of  Weeks  Two  arid  Three ^  all  participants  responding  to  the 
questionnaires  felt  1^e  material  presented  was  both  challenging  and  important. 
Virtually  , the  same  distribution  of  responses  was  found  for  an  indirect  question 
about  the  relevance  of  the  instructional  content  each  week:  vis.  "Would 
your  professional  peers  back  home  he  interested  in  what  you  learned  this  week?" 

■  Several  questions  in  the  weekly  opinionnaire  were  designed  to  elicit  ' 
participant  self-evaluations  of  their  learning  each  week.    After  all,  fjr 
participants  to  benefit  from  the  instructional  content  of  an  institute,  they. 
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must  master-  at  least  part  of  that  oontent.  The  first  of  these  questions  — 
"Did  you  learn  a  lot  this  week?"  --  yielded  the  following  responses : 


Yes     Yes  hut...  No  Don't  Know 

Week  One  1?  4  3  3 

Week  Two  13  3  6  2 

Week  Thvee  11  5     ■  [  8  2 

Week  Four  20  V      5  ,-  2 

Week  Five     '  '  2?  .       -  - 


(The  pattern  is  familiar  hy  now).    As  a  participant^  you  may  feel  you.  have 
learned  a  lot  hut  "Do  you  think  you  could  teach  another  group  what  you  have 
learned  this  week?" 


Yes  ^ 

Yes  but. . . 

No 

Don't 

Week  One 

11  \ 

10 

2 

2 

Week  Two 

■  ■  Ull 

3- 

1 

5 

Week  Tltree 

12\ 

7 

7 

Week  Four 

is: 

3 

3 

3 

Week  Five 

9 

12 

2 

2 

The  poPtioipants  are  telling  us  something  have^Yes^  they  have  indicated,  we 
have  learned  a  lot.    No^^  they  continued^  we  are  not  as  confident  that  we 
could  teach  it  to  someone  else.     (Week  Five  deserves  special  attention  in  this 
regard.    The  primary  instructional  activity  that  week  was  an  elaborate 
simulation  exercises,  one  which  would  require  much  greater  understanding  of 
content  and  a  not-insignificant  set  of  materials,  in  order  for :  anyone  to  attmpt , 
to  use  it  with  another  group.    It  is  not  at  alt  surprising  that  the  participants^ 
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who  were  unanimous  in  indicating  that  they  had' learned  a  lot,  were  much 
less  confident  tliat  they  personally  could  teach  what  they  had  learned  to 
others. ) 

Other  questions  on  the  weekly  opinionnaires  were  designed  to  get  at 
participants'  affective  reactions  to  the  week's  instruction.    Paralleling  the  - 
■earlier  question  about  participants'  ability  to  teach  the  week's  content  to 
another  group  is  the  question  which  asks  "Would  you  want  to  teach  another 
group  what  you  have  learned  this  week?"    By  week^  the  responses  to  this  item 
were:  '  ■    ,  ' 

Yes      Yes  but.. .      No    ■  Don't' know,. 
Week  One  26  5  5 

Week  Two  22  2  6  4 

Week  Three  '       22  2  8  4 

Week  Four  -     23  9  2  2 

Week  Five  18  4  2  2 

There  is  a  great  deal  of  agreement  between  responses  to  this  item  and  responses 
to  the  earlier  one.    There  are  a  couple  of  notable  exceptions ,  however. 
Participants  were  generally  uncertain  about  their  ability  to  teach  the  materials 
presented  in  Weeks  One  ( Cook)  and  Five  (Stafflebeam^  Guba,  and  Hammond 's 
simulation)^  but  in  both  cases  they  expressed  a  desire  to  do  so.    In  contrast ^ 
participants  were^  as  a  group ^  more  confident  that  they  could  teach  'the  material 
of  Week  Four  (the  Phi  Delta  Kappa  approach  to  evaluation)  than  they  were 
interested  in  doing  so.  -  ' 
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Another  approach  to  obtaining  affective  reactions  to  the  material 
presented  was  through  a  question  about  participants^  willingness  to  recommend 
that  the  materials  be  used  in  other  surrmer  institutes: 

Yes     Yes  but...      No     Don^t  know 
Week  One  19  4    \.  2 

Week  Two  6  7  7  4 

Week  Three.  13  6  4  3 

Week  Four  19  5  -  3 

Week  Five  26  I  -  - 

It  is  apparent  that  the  materials  from  Weeks  Two  and  Three  were  unpopular. 
How  much  pf  this  reactioji  should  be  attributed  to  the  participants^  negative 
reactions  to  the  consultant  for  those  weeks  is  of  course  impossible  to 
determine.    It  should  be  noted:,  however^  that  some  of  the  comments  We  received 
were  directhj  az'itical  of  the  materials  themselves.     (^^Turned  off  by  instruc-- 
tional  material.      ^^Redundant  and  uniAwportant .  ^'  ^^^Needed  revision  and  re-editing 

In  an  institute  characterized  by  a  '^parade  of  stars^^j  special  attention 
must  be  paid  to  the  problem  of  transition  from  one  topic  to  the  next^  from  one 
consultant  bo  the  next.    That  this  need  Went  unmet  inthe  New  Hampshire  In- 
stitute is  clear,    l^flien  asked  if  the  transition  from  previous  weeks  of  the 
Institute  was  satisfactory^  participants  at  the  end  of  Week  Two  (the  transition 
from  Cook  to  Walbesser  at  the  beginning  of  that  week)  were  evenly  split:  10 
said  it  was,  10  said  it  woB.  nc       Similarlyr  the  transition  from  PJeeks  Three 
to  Four  (from  P/albesser  to  Stuff  lebeqm)  was  seen  as  r  ^'satisfactory    by  15  partioi-- 
pants  and  as  ^'unsatisfactory^  by  10  others.    Throughout  the  Institute^  participants 
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were  uncertain  about  what  to  expect  from  each  new  oonsultantj,  oo?isultants  were 
unaware  of  what  their  predecessors  had  done  or  what  th^ir  successors  were 
going  to  do.    Once  again:,  the  administrative  staff  of  the  New  Hampshire 
Institute  must  be  ta    - .  '    task  for  failing  to  handle  this  very  importat^t 
aspect  of  the  Institute.     [Note:    Consultants  did  not  (in  gerieval)  share 
materials /outline  as  ^^prcmisedJ^  at  May  24^,  Washington  j  D.C.  conference.] 

Participants  were  asked,  at .the  end  of  the  Institute  to  make  some 
judjgments  about  the  relevance^  value^^  and  transportability  of  the  various 
components  of  the  Institute.    For  each  of  the  four  components  of  the  Institute^ 
over  three-fourths  of  the  participants  agreed  that  the  material  presented 
could  have  considerable  applicability  (this  in  spite  of  all  the  concern  about 
relevance)  and  that  they  personally  would  be  able  ■    put  the  materials  to  good 
use.    The  majority  of  the  participants  also  felt  that  the  matert.uli^  for  all 
components  except  that  involving  Stuff lebeann  (Weeks  Four  and  Five)  would  be 
^ ^'readily  trans f err able  to  other  Institutes  as  an  instructional  package  without, 
requiring  that  the  consultant(s)  who  presented  it  here  be  involved.  ^'  In 
direct  contrast  to  this  reaction^,  fewer  than  half  of  the  participants  ^'would 
recoimend  that  my  colleagues  attend  a  workshop  or  institute  which  had  this 
topic  as  its  central  focus  ^'  except  for  that  component  involving  Weeks  Four 
and  Five.    For  this  presentation  on  evaluation  models'  and  techniques^  all 
but  on^  of  the  participants  responding  to  the  Summary  Opinionnaire  would 
encourage  colleagues  to  attend  such  a  workshop  or  institute. 


THE  INSTITUTE  AS  A  WHOLE 

This  section  is  simplif  a  cataloging  of  some  of  the  svrmary  impressions 
of  the  participants  about  the  New  Hampshire  Institute,    Firsts  the  Institute 
achieved  its  objectives  ^'fairly  welV^  or  ^'very  welV^  for  nearly  all  partici- 
pants.   In  retrospect^  materials  in  all  ccmponents  were  seen  as  somewhat -to- 
very  relevant  (although  the  behavioral  objectives /learning  hierarchies  component 
was  rated  notably  lower  than  the  other  three  in  this  respect. ) 

Participants  also  felt  that  the  content  presented  was  both  practical 
and  feasible  and  that  their  competencies  to  conduct  evaluation  studies  and  to 
develop  and  'implement  sound  decision-making  strategies  had  been  enhanced. 
Finally^  after  all  ~  well^,  almost  all  —  was  said  and  done only  two  partici- 
pants indicated  thai  their  overall  reaction  to  the  Institute  was  unfavorable. 


..J 


APPENDIX  A 
PRODUCT  EVALUATION 


FILMED  ■  ROM  BEST  AVAIU\BLE  COPY 


Dr,  Daniel  Stuff lebeam,  in  a  program  mission  statement  developed  for 
Ohio  State  University,  makes  the  following  statements: 

"The  tasks  of  educational  development  include  ..efforts  1o 
understand  the  clynamics  of  educational  change,  to  identify 
targets  and  criteria  for  improvement  of  educational  systems, 
and  to  formulate,  test,  and  integrate  into  the  normal  opera- 
tion of  systems  improved  programs  or  ways  of  operating 
which  produce  higher  levels  of  systems  performance  ...  ^ 
Increasingly,  however,  educational  institutions  are  seeking 
highly  qualified  professionals  in  development  specialties; 
persons  with  broad  understanding  of  the  operations  of  the 
education  system  with  the  ability  to  understand  global  -J 
prooiems  of  large  systems,  such  as  local  school  districts  and 
state  or  national  educational  systems;  and  with  the  oapocity 
to  organise  and  operate  staff  activities  in  evaluation/ plan- 
ning, program  management  or  research  and  development. 
These  are  newly  emerging  roles,  not  yet  clearly  or  consist- 
ently defined,  which  represent  efforts- by  educational  systems 
to  institutionalize  mechanisms  for  orderly  systoms  develop- 
•  /    ment/'  V: 

It    is  to  the  training  of  these   kinds  of   people    that  this  Institute 
directly  addresses  itself. 


ERIC 


,       THE  INSTITUTE 

The  summer  Institute,  Planning  for  Educational  Decision-n,aking 
^n  the  70's,  sponsored  by  the  United  States  Department  of  Health, 
Education  and  Welfare  will  be  held  at  the  University  of  Nev/  Hamp- 
shire for  a  six  v^eek  period,  from  Jul,y\,6^to  August  14,  1970. 

Thirty-tvyo  participants  compriseci4pf  doctoral  and  post-doctoral 
condidates  from  throughout  +he  Un|tc?a  States  and  ils  territories  will 
participate  in  an  intensive,  cornprehensive  program  of  study  and 
training  in  the  methodology  pria  appHcotton  of  educational  research. 

The  Institute  will  cor-rcern  itself  with  the  problenis  of  training 
educators  at  the  doctoral  and  post-doctoral  level  in  the  skills  of 
project  management,  educational  research,  and  evaluation— as  aids 
in  the  process  of  educational  cacssion-making. 

The  major  themes  of  the  Institute  are: 

1,  The  study  of  the  problems  inherent  in  the  management  of 
an  educational  evaluation  project. 

2/  The  study  of  the  problems  inherent  in  evaluating  the  par- 
ticular curriculum  changes  wHh  vvhich  the  trainee  is  concerned. 

3.  The  study  of  the  major  alternatives  open  to  the  educator  in 
terms  of  educational  research  methodologies. 

4.  The  study  of  the  problems  associated  with  the  development, 
:  dissemination,  and  adoplion  process  in  the  orea  of  educa- 
tional evaluation; 

5.  The  study  of  communications  techniques  applicable  to  proper 
implementotlon  of  the  decision-making  process   at  various 

"    levels  of  the  educational  system.  : 

6.  The  use '  of  modern  data  processing  equipment  to  facilitate 
v      'the  encoding-  and  utilization  of  research  data. 

7.  The  reading  of  current  educational  literature  relevant  to  the 


ERJC  ■  research  project  with  which  the  trainee  is  involved. 


Program  and  Consultants 

■  Part  iciparing  in  the  Institute  will  be  ihe  fo!lowing  porsonnol: 
Director:  Albert  Elweil,  Bureau  of  Educcitionai  Reseorcfi  and  Tr::>ting 
Services,  Uriiversity  of  New .  Hcwnpshire. 

Consultants:  Desmond  Cook,  Educational   Program  Monagement 
Cont      Ohio  State  University,  and  Gregory  Trezebratowski,  Ohio  Stote 
■rsity^  who  will  lead  ihe  sessions  concerned  with  projecr  monage- 

Henry  Wolbesser,  University  of  Maryland,  who  v^iil  direct  trie 
study  of  the  construction  of  behavioral  obiectives  and  the  design  of 
assessment  tasks,  the  onstruction  of  learning  sequences,  and  a  study  of 
their  empirical  foundations.' 

The  third  section  of  the  InsMtule  will  focus  on  the  construction  of 
behavioral  objectives.  It  will  be  instructed  by  Dr.  Wolbesser  with 
the  assistance  of  Leonard  Cahen,  Educational  Testing  Service,  Prince- 
ton, New  Jersey;  and  William  Gray,  of  the  Maryland  State  Depart- 
ment of  Education. 

The  study  of  different  models  to  evaluate  education  will  be  di- 
rected  by  Daniel  Stufflebeam,  E /aluotion  Center,  Ohio  State  Univer- 
sity; and  Egon  Gubo,  Associate  Dean,  School  of  Education,  Indiana 
University. 

Dr.  Stuff lebeam  will  conduct  a  series  of  seminars  concerned  with 
problematic  simulation,  with  the  assistance  of  Robert  Hammond,  Ohio 
State  University,  and  Dr/Guba. 

The  final  sessions  of  ihe.  Institute  will  be  devoted  to  an  intensive' 
discussion  of  the  problems  associated  with  the  actual  development, 
dissemination,  and  adoption  of  educational  innovations,  directed  by 
William   Asher,   Purdue    University,   and   DanJer  Heisey,  Whittemore 
School  of  Business  and  Economics,^  University  of  New  Hampshire. 

Sarah  Curwood^  Chairman  of  the  Department  of  Sociology  at 
Q      Knoxville  College  will  act  as  a  speciol  consultant  to  Southern  -black 
EI^IC  .  educators  in  on  introductory  week  of  seminars.  ' 


HEW  Institute 
Planning  for  Educational  Decision  Making  in 
i  the  70's 

Application  Form 

.(detach  and  niail  before  June  1,  1970) 


name 
address 


zip. code 

home  phone  _  

age  ^    ^        u.s.  citizen? 


number  of  dependents  (excluding  yourself)  wh 
claimed  on  your  last  federal  income  tax  return  (you 
may  not  claim  any  dependents  if  you  filed  a  joint 
return  and  were  not  the  major  wage  earner) 


are  you  presently  ernployed  in  a  school,  system,  or 
college  and/or  employed  by  title  I  or  title  III? 


ERIC 


no 


(if  not  presently  employed  in  a  school,  system,  or  col- 
lege and/or  employed  by  title  I  or  title  III,  omit  the 
items  on  this  page,  go  on  to  next) 


school  name 
address 


zip  code 

school  phone 

type  of  school  (technical  institute,  elementary,  junior 
high,  etc.) 

number  of  students  enrolled 


name  of  supervisor 
your  position 


if  you  ore  preparing  for  a  different  assignment; 
specify  here  . 


summarize  your  years  of  teaching  experience  and 
related  work: 

subjects  or  assignments  level  years 


list  your  places  of  employment  in  teaching  or  related 
work  in  the  past  five  years: 

place  address 


V  what  colleges  or  universities  have  you  attended  (most 

recent  first):  . 

date  Institution  degree  major 


list  all  undergraduate  and  graduate  courses  in  eval- 
uation and  measurement: 


what  teaching  certificates  or  other  credentials  do  you 
hold? 


describe  any  other  significant  academic  experiences 
you  have  had  in  the  subject  field  of  this  summer 

institute 


list  the  professional  organizations  in  which  you  hold 
membership 

are  you  applying  for  any  institutes  or  fellowships  in 
addition  to  this  one? 

.  if  yes,  sp  -  ify  '  .  ^ :  ".      /.     'y'-'-i  : 

I  certify  that  the  statements  made  By  me  in  this  appli- 
cation are  true,  complete,  and  correct  to  the  best  of 
my  knowledge  and  belief,  and  are  made  in  good 

faith.  :-:,r::''--^ 

date  signature  :  - '  \  :   ..       ^  ^ ;  .  '  .  . 


Academic  Credits 

Six  hours  of  graduate  credit  will  be  aworded  for  the  successful 
completion  of  the  Instifute.  The  participants  may  apply  these  credits 
at  the  doctoral  or  post .  doctoral  level-Credits  wilhbe  awarded  ,  ac- 
cording to  the  following  prospectus^ 

Education  881,  (4  credits),  Methods  and  Techniques  of  Educational 
Research.  This  course  is  a  critical  study  of  the^  principal  methods  em- 
ployed in  the  Investif-vNon  of  educational  problems  and  an  evolucition 
of  the  procedures  u-  ;'  standards  uied  in  reporting  the  findings;  de- 
signed 05  cm  advanceCv, course  for  candidates  for  graduate  degrees. 

Education  895,  (2  credits^  Research  P«roblems  in  Education  is  con- 
cerned with,  the  individuo!  investigation  of  a  problem  in  the  areo  of 
educational  research.  This  course  will  be  i:;sed  first  to  expose  the 
trainees  to  a  variety  of  educational  research  efforts,  In  the  latter  por- 
tion, each  trainee  will  be  expected  to  develop  o  research  proposal 
which  will  have  procticol  importance  and  relevance  to  his  home  district, 
or  to  evaluate  a  project  presently  being  studied. 


Stipends  and  Fees 

TuiJion  and  fees  for  participating  members  of  the  Institute  are 
provided  for  under  the  accords  of  the  HEW  grant  for  fhe  summer 
program. 

Following  o.'f.eptance,  participants  may  receive  a  stipend  of 
575.00  per  v^eek  plus  $15  per  v^eek  for  each  dependenit  for  the  period 
of  attendance  ot  the  Institute^ 'Stipends  will  be  paid  bi-vvee.kty,  on 
July  17,  31,  and  August  14.  A  maximum  of  2/5  dependents  will  be 
guaranteed  payment  throughout  the  six  weeks  of  the  Insiitute,  Addi- 
tional dependency  payment  wil!  be  determined  by  the  amounr  of 
funding  available.'  ■ 

Participants  will  also  receive  travel  expense  payments  for  one 
round  trip  between  their  home  and  the  University  of  New  Hampshire 
Textbooks  and  supplies  normally  associated  with  graduate  programs 
will  be  purchased  by  the  enrollee. 

Since  the  success' of  the  Institute  will  depend  on  the  /sharing  of 
ideas,  "participants  will  be  housed  in  one  University  residence  hall 
and  will  have  fheir  meals  in  a  reserved  area  of  the  dining  hall.  Par- 
ticipants will  be  requested  to  avail  themselves  of  these  facilities  Mon- 
days through  Fridays,. and  will  be  responsible  for  their  costs.  The  total 
charges  for  the  six  week  period  are  as  follows; 

:  Single  room  and  15  meals/week  $228.00  * 

20  meals/week  264,00 

Double  room  and  15  meals/week  210.00 

20  meals/week  246.00 

,A  private  Institute  reading  room  will  be  made  available  in  the 
residence  hall. 

Criteria  for  Admission 

Institute  applicants  must  be  presently  working  in  an  approved 
Doctorate  program  in  the  area  of  Education  and/or  hold  on  earned 
Doctorate  in  the  area  of  Education. 

The  applicant  must  also  meet ,  one  or  more  of  the  tollowing 
criteria: 

1.  He  must  be  one  of  the  personnel  in  any  State  Department  of 
Education  in  the  United  States;  having  responsibility  for 
school-related  projects,  or 


■  I' 

2.  A  curriculum  coordinator,  such  as  an  Assistant  Superintendent 
of  Schools,  who  has  been  designated  a  pro'ect  director; 

3.  '   A  member  of  Senior  H'  ,  h,  Junior  High,  or  Elementary  School 

staff  responsible  for  implementing  icurriculum  study; 

4.  A  member  of  a  college  or  university  faculty  or  staff,  serving 
as  a  research  consultant  to  any  public  school  system; 

5.  A  staff  member  of  a  public  or  private  school  interested  Jn 
developing  research  competencies; 

6.  An  employee  of  an  eclucational  research  loborotory  or  a 
research  and  development  center.  ; 

Acaden)ic  Criteria:  The  opplicont  must  possess  o  Bachelor's  degree 
from  on  accredited  institution,  evidence  of  satisfoctory  scholarship, 
and  promise  of  ability  to  succeed  at  the  graduate  level.  An  under- 
graduate grade  point  average  of  '^..5  (on  a  4.0  scale)  will  be 
minimum. 

Teaching  Experience:  A  minimum  of  tv/o  years  in  public  or  pf^ivate 
schooi  education  and  evide-nce  of  ability  to  provide  leadership  and 
responsibility  in  the  orea  of  curncul.'./m  reseorch  will  be  required.  A. 
Jetter  of  recommendation  will  be  requested  from  the  applicant's  im- 
mediate superior  indicating  the  meeMng  of  these  criteria. 

The  Institute  wil!  operate  in  compliance  with  the  Civil  Rights  Act 
of  1964,  which  states:  "No  person  in  the  Uniled  States  shall,  on  the 
ground  of  race,  color,  or  national  origin,  be  excluded  frorri  participa- 
.  tion  in,  be  denied  the  benefit  of,  or  be  subject  to  discrimination  under 
any  program  or  activity  receiving  Federal  f inanciol  assistance."' 

Application  Procedures 

Appliccnts  to  the  Ihstituto  who  fit  the  above-mentioned  criteria 
will  be  selected  for  admission  by  a  Selection  Committee,  compr*sed  of 
individuals  directly  concerned  with  teacher  preparation  and  research 
in  education  at  the  University  of  New  Hampshire. 

Applicants  for  admission  to  the  Institute  inust  return  the  enclosed 
.  Application  Form  to  the'  Bureau  of  Educatio.ial  Research  and  Testing 
Services  no  later  than  June  1,  1970.  The  Confidential.  Evaluation  Form, 


to  be  completed  by  the  applicant's  immediate  supervisor  must  also  be 
returned  by  June  1 . 

Accepted  applicants  and  alternates  will  be  notified  by  June  15. 

A  letter  of  acceptance  from  the  selected-  applicants  and  alternates 
must  be  forwarded  to  the  Bureau  of  Educatlonol  Research  and  Testing 
Services  no  later  than  June  22. 

Recreational  Facilities 

The  University  of  New  Hamp^jhire  .is  located  in  the  southeastern 
part  of  the  state,  near  beaches,  lakes,  mountains,  and  resort  areas. 
On  -campus  facilities  include  a  modern  student  union  building,  swim- 
ming pool,  tennis  courts,  and  fully  equipped  field  house.  1^  addition, 
the  University  sponsors  a  summer  theater,  concerts,  recitals,  films,  and 
other  events.  Within  'i>olking  distance  are  shops,  restaurants,  ond 
churches.  The  cultural  centers  of  Boston  ore  less  than  70  miles  away. 

A  v/eekly  recreational  scfiedule  will  be  posted  for  Institute  mem- 
bers; group  activities  may  be  planned  according  to  interests  expressed. 

Calendar 

June    1,    1970.    Application   Form  •  ond  Confidential 
.    Evaluation  Form  to  be  returned  to  the  Bureau  of  Educational: 
Research  and  Testing  Services,  University  of  New  Hampshire, 
Durham/  New  Hampshire,  03824. 

June  15,  1970.  Notification  of  accepiance  to  be  sent  to 
the  accepted  applicants  and  alternatives. 

June  22,  1970.  Letter  of  acceptance  from  the  selected 
applicants  and  alternatives  must  be  forwarded  to  the  Bureau 
of-  Educational  Research  and  Testing  Services. 
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SOUTHERN  BLACKS  AND  THE 
NEW  HAMPSHIRE  1970  SUMMER  RESEARCH  TRAINING  INSTITUTE 


Report  Submitted 
hy  - 
Dr.  Sarah__Curwood 


«wv.v»*w*«.  w4.«v-A.o  axjLu  w.n,  HiiH  insxLxute  1970 

R^T  ^"'^^i^L^'Tf^^''  HEW  institute:  directed  through 

a  ?1        university  of  veir  flampshlre  during  the  past  three 

ch^^i^  i     ^^/^^^  sood  for  New  ISngland  educators, 

th^  jeed^f^^and  good  for  southern  Black  educators,  accept- 

the^auxtitude  ox  la.cics  produced^^^^^  standing  segregated  school 

^Although^my  Initlal^concern  with  the  black  group  to  attend  the 
?,,°f^?®'^®^.^°''??  orientation  to  an  anticipated  new  cultural 

educational  qualifications  grew  dominant 
le  recruiting  progressed,  uuuxxxttuw 

-d^by.hurdling_^!nany  obstac 3,63.    Pew  black  Institutions  conferred 

:„  had^to  be  done  away  from  home,  usually  at  a  new  Inst i- 

2^L°f  ^-.^  predominatly  white.  Here 

idequaoy  of  their  undergraduate  preparation  was  critically  asisessed. 

studying  done  for  the  master's  degree,  especially 
lucatijjn,  was  confined  to  the  summer  period.    -Doctoral  degree 
•ams  with  more  stringent  residence  requirements  often  meant  an 
noting  of  the^famlly  with  the  usual  problems  of  black:  relocation 
temporary  splitting  up  of  the  family  during  the  period  of  resl- 
The  reyealed^and^the  hidden  costs  program  were 

i^  not  /Justified  by  the  Job;  oppd^^^  in  the  candi- 

■         ;°0Ji^uiiJ-ty  or;  elsewhere  in  the  South  or  in  the  JJorth. 

^^spite  cif  these  ya^ 

ssfuily  ,cofiipleted  a  doctoral  program,  he: would  tend  to  move  into 
r; .-.education.  ■.v'^'' 

the  credential  game, ^^^^^^b  to  presen  t  ■ 


e  .following  :ls;^^m^^  brlentation 
d  ;for  the;  Southern  ■black  participants  in  the 


a; 


1;.  V  Toi^ailbw  the.:^^^^^  acquainted  with 

each  other  without  the  presence  of  the  whites  and  the 
;  -  ■;,  -.^pressures :  of  ■■learning.;;? 

2.  -  :-.;;To  ■.■allow.  ;ihem...';::::;'-:^ '';:--■■■  ■'■'■■^"^^  ■;•';■'■• 

ah  opportunity  to  relate  to  resident  -bl^^^^ 


f^d; see-t^ 

■;(.,3oe:;:llsT)V:-:;v-C:-v"-^v  .■■■•,.-  ^ 


through  their  eyes. 


3.    Jo  have  ah  opportunity  through  informal  gatherings  to  expres 


To  have  an  opportunity,  through  t-nl^snraal  ga-t-hora^gs  to  exurees 
concerns  relating  to  the  program  and  their  own  situation. 

.     To  build  mutual  trust  and  cionfidence  between  the  individuals 
and  the  staff  of  the  Institute. 

.     To^ward  off'a  "blaolc  caucus  of  protest"  that  often  occurs  in 
integrated  seistings  that  intentionally  or   .mintentionally  fall 
to  meet  the  needs  and  abilities  of  black  participants. 

.     To.  establigh  a  social  climate  that  would  maximize  learning 
and  mutual  Interaction  and  sharing. 

llowlng  flexible  schedule  was  planned. 

First  day:  Wednesday  afternoon,  arrival  and'  settling  In" 

Wednesday  evening,  first  orientation  session  at 
Stokes  Hall— -  discussion  of  New  England  and  ' 
,  Hew  England  life,  with  special  reference  to  New  Hamp 

Second  day  J  Thursday,  brea:k:fast  at  the  Ourwood  Homestead. 
Emphasis  on  getting  acquainted. 

Thurs,  afternoon  -  meeting  on  campus  with 
?'lrs  ,  Adams    ,  Oi\:  i^      V  ..i.- jT  ^ 

I-dr.  Johnson,  adxl  S3  ions  officer  for  black:  students 
Discussion  of  black  students  on  campus. 

Evening  ffi^^^^ing  with  black  members  of  the  wider 
com nunity/reGreat ion/theater  etc.  . 

Third  dayr    Friday  morning.  Individual  Interviews  with  Ourwood 

^'-'^i^^ay  iifterno^o  with  HEW  Institute 

■"staff  .members 

"-.Evening;  free  ,  , 

Fourth  dpjfi  Saturday ,  morning  s  Individual  Interviews 

>    Saturday  afternoon:     Area  sight-seeirg.^^^^^^r^^  ; 

Saturday  evening :     BERTS ,  institute  staff  and 
•    ■participants  picnic  and  supper  at  home  of  Jan 

;:and;-lierb";:Scheibel'v  ■  ■  v:^  ^ 

Fifth  day  t    Sunday  morning:  Ohurch  -  slgSiiseeii^ 
^■:  and afternoon 

Eveningr   VfelcomS:2ig  of  other,  members  of 

■■■■■■;.;;  :/■■  .  Institute. 


The  Initial^  plans  for  orientation  were  seYerly  modified, 

1/    The  lateness  of  recruiting  early  showed  that  a  full  work 
5®''^°^  orientation  was  impossible.     The  proDoae 

foiJth^Jf^J^^^^  mclude'Sfia 

2.  In  spite  of  the  later  starting  date,  all  of  the  part- 
~?:£f^'*^f  <^^'^  ^o''^  ^^2?ive  on  time  and  some  did  not  arrive 
within  the  dlsignated  period  when  individual  interv*  etrs 
wl?e  inte?^ilK9d.°^^^  ^^"^  """^  the  ten  hlaok  partic  Ip^ts 

3.  rwo|-black  participants,  not  part  ot  the  ten,  and  one 
patlci pant  from  Guam  parti r^ipated  in  the  full  f ive  ■ 
day  period, 

Wednesday  evening  four  persons  met  with  me, 

Thursday  morning,  five  oame  for  breakfast.     This  was  a  very 
laaant^  experience.     "who  knows  whom"  is  an  accepted  and  important 
slgnatlon,  i.n  .Southern  black  groups  tha4;  are  criss 
leges  attended ,  mutual  friends ,  kinship  and  fraternal ^ 

tlcipants  was  Shared  informally. 

P^lday's  day,  awaiting  the  arrival  of  more  members,  favored 
eral  informal  relationships  with  Institutional  staff  members. 

I  Interview  two  members.     Saturday  morning 

ntervlewed  two  more.     The  picnic  supper  was  held  at  ray  place, 
rlendly  cooperative  spirit  prevailed  as  the  five  black  members 
®  Joined  not  only  by  he  Institute  and  BERTS  staff  members,  but 
o  by  other  early  arriving  white  participants.     I  rejoined  the 
up  Sunday  afternoon  in  time  to  welcome  more  blaok  participanta 
_ to  interview  two  more*     During :  the  institute,  I  met  the  rest 
the^black^participants  ?  took  some  sight-seeing  in  Portsmouth, 
I  did  not  nave  an  opportunity  to  complete  the  interviewing . 

fet   potential; /guests.,  /  '  V:  '  ' 

Lena  Ool<smant    Long  time  resident  of  Concord,  K,H» 

Mel^ln  Boidenj    Deck  Road,  Loudon,  in. H.     Politically  aot ive 

in  the  Democratic  party. 

The  Bacon  Brothers i    of  Pembroke p  owner  of  a  rug  cleaning  plant 

in  Concord,  N.H.    Mrs,  Bacon  works  for  the 
:     S'amily  Service  Agency  in  Concord. 

Clifford  Lawrence  of  Andover,  Mass.:    works  with  the  Deaf  and 

Blind  Ofeildren's  Division  in  Concord. 


rhe  questlonaire  that  the  partioipants  filled  out  sought  baok- 

I  information  about  the  participant fi  spouse  and  riarents  to 

gome  small  understanding  of  the  present  home-educational  back- 

*  ^<i  the  possible  childhood  home-educational  backgroiind  of  an 
engaged  In  education. 

Ehe  Interview  schedule  must  be  vievred  as  a  Draliminary  form 
^  at  generalized  areas  of  conoern  that  I  thought  might  be  rel- 
to  help  us  understand  the  Interviewee  during  a  "committed  to,  but 
Jtually  in"  situation  as  a  base  line  for  later  comparisons. 

Phe  questions  ,  r  believe  J  are  relatively  3elf^-explanatory, 

Phe_f laal  sheet  on  which  social  inte:.a&.4-i ok^^*  w--^  recorded  has  - 
tlal  for  usefulnessp  but  heeds  an  in^  ^c-coo-d   .^i?  .r-nvcT 

The  six  interviews  came  from  four  staV-e^  '.  Two  from  Tennessee, 
:om  Louisiana  (north  -  south) ,  and  one. e^ch  from  Mississippi 
»xas.     Three  of  the  interviewees  were  nicires,  three  females,  one 
>t  married o     Most  (A)^  were  between  the  ages  of  42  and  47.  One 
rer  fifty  -  the  other  under  30.     To  round  it  out  the' average 
18^42.  ,  .  / 

All  Interviewees:  are  basically  stabiles.     iuLl  are  living  in  tha 

where  they  wire  bom j  all  received  undergraduate  education  In 
■home  state,  half  of  the  group  did  graduate  study  In  their 
statQ.     of  the  five  vrho  were  married i,  four  of  their  spouses 
also  born  and  educated  in  the  Home  state, 

ut  one  held  master *s  degrees ;  four  of  them  in  education,  one  in 
ry.     One  husband  completed  high  school  and  one  college.  All 
wives  have  college  degrees      one  a  masters*  . 

al  of  9  children  belong  to  the  five  who  are  married;  one  has 
children,  on©  tec  and  the  rest  have  only  one  child. 

The  Interviewees  represented  a  rather  immobile  group  oif  persons  - 
educ at eid  vri thin  their  respec t i ve  c ommunlties ,    The  men  are  married 
liege  graduate -wives.     The  family  size,  with  one  exception,  the 
family,  is  very  small,  but  rather  typical  of  the  southern 
eolass  Negro  family,  long  represented  in  "school  teaching". 

The  educational  ac hi evcmant  of  parent Ss  in  general.  Is  of  a 

r  level  than  that  of  their  children.    Only  two  of  the  Intar- 

es  had  fathers  who  had  completed  college j  only  one  had  a  mother 

ad  oompletecl  college.    One  had  a  mother  who  attended  college 

wo  years,  but  whose  father  had  completed  high  school.  The 

tlon  of  the  other  four  fathers  was  Tth  -  8th  (2),  4th  (1),  D£  (1). 

©  other  four  mothers,  one  completed  high  schools  (2)  7th  -  8th. 

ae  DK. 
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If  the  five  spou8es*  parents 5  three  faifchers  and  two  aothera. 
were  DK*s,  two  fathers  and  thr<?o  mothers  were  7th  grade?  the  "ohildren" 
have  nade  big  leaps  compare'd  with  their  parents,, 

#1    All  but  one  of  tho  Interviewees  saw  themselveB  as  having  been  reoruite 
for  the  prgram       a  blacke    The  single  one  directly  attributable  to  a 
whit©  reoruite::  oa^na  about  as  an  aocident<>    (I  laintock  Texas  A  &  M 
(predominantly  rhite)  for  Prc.irle  irievj  (predomlnan'':ly  black).) 

IPVfo  to  tbr     contacts  -  rather  chain  effect  wer  used  to  recruit. 
The  proce^ss  Tras  much  more  time  consuming  than  I  had  anticipated,  but  ■ 
I  feel  Tre  opened  up  nsvr  channels  for  fo^ture  training.    Much  of  what 
"happens"  secerns  not  to  filter  through  to  blaol':  Gd-ucatorSe 

Pour  of  the  participants  tvoro  reorui ted  mainly  through  college 
contacts  I  t>ro,  thi^ough  atata  departments  of  education, 

#2    All  were  pleased  to  be  ooming  to  the  institutse,    For  the  most 
part,  It  wa,B  an  opening  up  of  an  linkno-vm  section  of  the  cotmtry. 
Only  one  expressed  mised  foelingn  of  being  maneuvered  to  come  to 
used  as  a  "symbol" 

#3    Most  cause  professionally  to  learn  and  start       continue  graduates 
work:  In  a  mora  challQngln^  environment,    several  were  encouraged  by 
supervising  persons  j^o  oome« 

Personally  most  had  a  curiosity  about  a  new  section  of  the 
country,    one  had  made  plans  for  his  family  to  share  in  the  experience 
the  last  ire ek  -'joining  work  and  recreai*;lon« 

#4    None,  expect  few  logistic  details, 

#5    Xhe  shortness  of  the  time  between  being  accepted  and  having  to 
comQ  caused  tLs  Biost  problems  «  ft^Z"  some  it  meant  release  from  regular  vyi:- 
summer  obligations 9  for  one  an  actual  salary  lostj  and  failtire  to  oomplet^ 
one  more  course  f or  eertif ication  (9  Hew  Hampshire  creelits  are  not 
wotth  anything)  .■™  in  this  case,    For  one  without  charge  cards,  the 
aotual  money  for  travel  was  a  big  hurdle «    The  trip  for  the  most 
part  was  V93ry  long  and  ex^isustlng.    The  loast  number  of  problems 
were  faced  by  those  participants  who  knew  In  April  that  they  would 
probably  come, 

#6  '.No  one  se'emsd  to  have  a  clear  picture  of  what  the  experlenoe 

would  mean  when  they  returned  1 ,  e ,  no  clear  rosponalMlA ty_  "ta^^  t^  baplc" 

have  been  on.  the  job.    He  oemo  with  the  blessings  of  an  active 
superintendent  who  could  forse©  benefits  to  the  system  from  this 
oxperlenoe,    one  intsrvieweo  is  responsibla  for  making  a  full  report 
to  the  State  Office  of  Education  (black  sector)  and  to  talk  to  local 
groups  about  the  experience* 

No  one  anticipated  any. problems  when  they  returned  to  home  base. 


^^:,S^^^        i^ember  seemed  fully  self-assured  about  being  in  the 
program.    One  covered  Q elf  assurance  or  anxlaty  or  both  In  a  global 

i^^A^'^^^T^^^^^^^  x.et  expressed 


^  thev  Sn?f  v^""  and  ^degree s  .  strong  an>:i et i o s ,  pr inc ipally  that 
1  pushed  too  hard,  not  allo^red  enough  time  to  grasD 


alnst 


anxiPtv      r>o-^+J^       -   1'^^  participant's 
■:.-learn!       ^'Q^;^^^^  -^^ar  too  of  the  GOMFUTSR.  Eut  all  vers  willing,  to. ; 

yM  -ynapeQined  research  skill s  and  computer  skills  were  senerali v 

theifore'f^^thP~^ti'^^^  evident  l.n  various  ways  caaie  to 

..  .  .xne  rore  in  fche.  auswer^  ^o.    The,  most,  estramesr  -  ■ 

■  -'^^^  don't  use  it  against  me.     People  here  raav  be  okay  but  it's 

:    i-ederal  material  and  the  next  per soxt  niight  LHe  okir  ' 

■b.    don't  use  it  to  separate  blacks  from  whites 
v;,V'-'\;-y.:0.    don't,,' degrade,. 

Several  ^generalized  that  "that  is  the  way  it  should  be,"  i.e. 
testing  at  any; time.    Overall^  they  viewed  the  testing  as  "partof  the 
game,  part  of  the  price  they  would  have  to  pay, to  bein  the  program. 
(Resignation  to  fate  I) 

#11    The  overall  picture  on  intergati on  — • 

12  ■  •  , 

Sometimes  a  black  teacher  is  offered  only  one  job  -  if  she  refuses 

■.  :     .m.m.m,  -out, I 

^  are  resigning,  but  have  other  Job  options  blacks 

do  not  possess,  ■ 

If  intergration  is  happesning  at  all,  it  is  to  replace  good  black 
teachers  with  poor  -  less  qualified  white  teachers.  Non-tenure 
•teachers  may  be  moved.  Theoretically  teachers  have  a  oho Ice,  but 
actually  it's  a  Hobson's  choice, 

;    Plve  were  quite  mindful  of  intorgratlon  problems  and  relatively 
:  :     knowledgeable  about;  them  even  when  basically  in  a  segregated  sit-  ' 
;    ;   uatlon.    Only  one  seemed  to  Just  accept  the  status  quo. 

^  ^  r?Jl^^thl®S+&fi^  only 
one  real  -  the  other'  ono  was  ''paper  supervision.  ^ 

^t3    Integration  had  affected  perDonally^ 

warm??ii^?r^^r^^  one 


#14;  Inter-raclal  contacts  la  South 


1 


?ple  only.  -  •wiiite  —"  limited  area 


,:  9  ■  •'•.■.Hjjorlenc es ■ 

a.  :  Educational  meetings  -  white  biacic 

b.  Black  friends  in  Boston  and  Massachusetts 

c.  Integrated  church 

d.  Studying  at  "white"  institution 

e.  school  and  political  contacts  ( money  from  blaclc  teacher 
for  UAAGP) 

f .  Suliensrlsor  in  army  school  --.  wide  contacts  —  black  -  white 

g.  6  week  institute  N.y.  teacher  coming  in 

health  program  — not  for  BLAQKS 
w  hit e  grad  school  family  white 


In  general,  wider  contacts  with  whites,  northern  blacks  seemed 
to  depend  largely  on  out  of  souths  c-oiitac  ts 


It       Oontacts  maintained  with  migrating  blacks. 

2.        Sufficient  ranking  in  educational  system  to  be 

allowed  to  attend  wide  range  of  professional  meetings. 


Broading  experience 


a*  arziy  attached  to  educatiorxal  unit. 


graduate  study  in  white  Institution 


Job  experiences 


-  southern  whites. 


ERJC 


1.  How  did  you  hear  about  the  institute? 

2.  What  was  your. Initial  reaction? 

3.  ;  '  People  came  to  Institues  for  a  variety  of  reasons: 

,  a)  V/ht.b  ¥as  your  ma;)or  prof easional  reason? 
b)  I'/hat  was  your  major  personal  reason(s)  ? 

:  4,      vrnat  influence  did  personalised  contacts  with  administrators 
of  ;the  Institute  have  on  your  declaion? 

5.  What  problem  situations  did  you  encounter? 
■ ;  ■  ■■       release  ■ 

b)  transportation 
0 )  f Inancee 

6,  What  vrill  be  expected  of  you  after  your  return  from  the  Institute? 
a)    from  your  employer? 

bi    from  yourself? 

c)  from  your  family? 

T.      Do  you: anticipate, 

a)  :  working  relationships? 

b)  inter-personal  relationships? 

8,  "     What  anxieties  do  you  feel  about  the  adequacy  of  your  back- 

ground for  suocess  in  the  program  as  you  now  understand^^^^W 

9,  :  What  are  the  skills  and  knowledge  you  now  peraeive  as  essential ? 

10,  How  difficult  is  it  for  you  to  view  a  testing  program  aa  a 
diagnostic  tool  for  small  group  planning  rather  than  as  a 

^  Judgment  as  personl  deficiency?  . 

•11,    What  is  the  condition  of  schools  integrated  in  your  area? 
12.  ,  How  Is  It  affecting,  black  teachers  and  black  administratSSs? 
.13.  ;  How;  Is  it  affectiiig  you  perlbnally?  : .  ^ 

14,    Some  peop3.9  believe  that  degrees  of  intergation  between  whites  and 
blacks  are  essential  as  ^^k^  dispersed  from  center  of 

understanding  such  as  this  University  through  this  Institute,  ■ 
While  mfereroontacts  between  diverse  seeking  comraon  knowledge  are  not 
a  panacea  i'or  our  soo Val  Ills.    The  absence  of  contacts  bl^ojcs 
even  thd'potenlal  for  understanding.    I  am  interested  in  the 
.experiences  of  social  Inter-action  you  have  had  with  southern  and 
northern  whites  and  southern  and  northern  blacks  in  various 
regions,  and  un.der  varying  circumstances. 


PRE-TBST  AND  POST-TEST  FOR 
PLANNING  FOR  EDUCATIONAL  DECISI0N-MAKIN6  IN  THE  70 'S 


I,  Fundamentals 


1.  The  most  marked  difference  between  V'basicV*  and        lied' '  research 
in  education  is  in  the  extent  to  which 

a.  control  and  prec  sion  axe  einphasized. 

b/  the  Hawthorne  effect  operates . 

c.  implications  are  drawn  concerning  school  practice. 

d,  theoretical  constructs  are  tested. 

2.  The  major  difference  between  p^^ 

a.  use  of  research  design. 

b«  iiiethods  of  population  selection. 

c.  stability  of  conclusions  reached. 

d.  application  of  the  findings. 

3.  Which  ono  of  the  following  defects  is  almost  iii5)ossible  to  reioedy 
in  the  process  of  prob lem-s61ving  through  personal  e^cperience ? 

a*     Subjects  and  situations  are  probably  not  typical, 
b*     Assuntptions  are  not  identified. 

c.  Meanings  are  not  defined. 

d*     Processes  of  solving  are  not  replicable. 

4.  Which  one  of  the  following  "methods  of  knowing"  is  most  characterized 

by  self-correction? 

a,   ■ .  tsnac.i.ty 
b-  authority 
c»     a  priori 

d,  scientific 

5.  Which  of  the  following  is  the  least  importaiit  aim  of  •»pur«"  research? 

a.  control 

b.  prediction 

c.  sys temat i z  at  ion 

d.  !  application 

6.  The  aim  of  theory  is  the  explanation  and  prediction  of  phenoweiia, 

■  ■a. ,    True  /' 
>•  False 


7  •    I-Zhich  one  of  the  following  best  describes  induction? 


a.  Spi..ci.fxc  to  general. 

b.  General  to  specific. 

c.  Specific  to  specific, 
d*  General  to  general. 

Read  the  quotations  in  items  8-10  below.  What  source  of  knowledge  is 
represented  by  each  statement?    Use  the  following  key:        .  .  ^  '    ■  ..  . 

a*  authority  ^: 

b.  inductive  reasoning 

c.  deductive  reasoning 

d.  tradition 


8.    Mrs .  Adams  said       know  that  the  children  in  the  Head  Start  program 
have  been  given  considGrable  practice  in  reading  readiness  skills. 
Since  it  is  known  that  practice  improves  these  skills ,  it  figures 
that  those  children  who  have  participated  in  Head  Start  will  be 
superior  in  reading  readiness." 


9.    Kiss  Brown  replied,  "I  don't  believe  that.    Our  principal  says  the 
Head  Start  prcg-iram  did  no  one  any  good.    Therefore,  I  expect 
the  Head  Start  children  to  be  no  better  on  reading  readiness  than 
tlie  other  studt^nts." 


10,    Mrs .  Carter  said,  '^Let  us  examine  and  measure  the  children  on  reading 
readiness  in  evsry  way  we  can  to  determine  whether  the  Head  Start  children 
do  better  or  not  : 


11.    One  difference  between  the  experimental  and  descriptive  methods 
in  research  has  to  do  with  the 

a.  setting  of  the  study. 

b,  control  of  variables • 
number  of  subjects. 

d.  recording  of  data. 

e.  time  span  of  the  study. 
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The  most  important  characteristic  of  the  experiraental  v-.etliod  is 


a.  ths  repetition  of  observ-ationa - 

b.  the  control  and  systeinatic  variation  of  the  ronditions  of 
observation.  * 

c.  the  Hiaking  of  exact  :neaf3urement.s . 

d.  the  concapt  of  correlation* 

13*    An  investigator  was  interested  in  studying  teacher'- success .  He 
selected  txvfo  groups  of  teacheri^ :    one  that  was  highly  successful 
and  cue  that  was  not  too  succesftful.    He  found  that  the  luore  successful 
teachers  tend  to  be  more  out-going ^  mere  independent,  and  more 
interested  in  people  ^than  jess  successful  teachers.    The  type  of 
design  used  in  this  study  v/as 

a,  experimental. 

c.  longitudinal • 

d .  correlational . 
e*  cross-sectional* 

■;■  '  ..       -    y  ■  ;■  ■,  ^         ; ;.    ;  ■.  .  ■,     p  ..  . 

Midland  Central  School  District  has  recently  adopted  the  "new  math" 
into  its  program  at  the  7th  grade  level.     It  is  now  interested  in  a 
study  of  6th  grade  students  In  order  ^ to  evaluate  the  effects  of  the  new 
njethod ;    One  hundred  students  vrere  included  in  the  study  ,  25  from  each  of  / 
the  four  elementary  schools  in  the  district.    The  25  students  from  each  schoo 
were  identified  and  included  _ in  the  study  in  such  a  way  that  all  students 
in  the  6th  grade  in  each  ^school  had  an  equal  opportunity  to  appear  in  the^ 
25  from  each  school.    The  .total  of  100  students  were  then  placed  into  three 
testing  groups  by  a  method  which  assured  that  there  would  probably  be  no 
differences  between  these  three  groups  in  terms  of  extraneous  variables, 
;This  was  done  so  that  three  different  forn'is'  of  a  standardized  achievenient 
test  could^  be  lised  "'and  compared.    The  students  xvere  then  administered  the 
testsr    "Hie  results  were  analyzed  and  conclusions  drawn  concerning  the 
success  of  the  new  method  with  6th  graders  in  the  school  district.  V 

14.  Random  selection  was  involved  in 

a.  placing  students  in  each  of  the  three  testing  groups. 

b.  administering  tests  to  the  three  testing  groups. 

c.  identifying  25  students  from  the  6th  grade  la  each  school. 
d»  deciding  to  use  modern  math  as  an  instructional  rnethodc 


Suppose  a  re^iearcber  wanted  to  deder:nine  the  effects  on  the  attitudes 
of  citixcsiS  tc^t/'&rd  cent r;:Xlii?.4^d  .  gc^^^  of  the  electrical 

hlQi±o^n£  ^>Axi^^^^^  In  Northeastern  United 

States .    He  hypo?:lia£:ii:od  th!DA:  those  peep  la  living  i/ithln  the 
blackout  area  vrould  be  move  ovvpocf.id  to  rv;deral  cont3;ol,  since 
a  ?ederal3.y  iuiti.ated  Rrid  sy£ten  allowed  the  hlaclcout:  to  spread 
from  it55  origin  and  since  these  people  were  actually  affected 
by  tlie  blackout,         prelected  a  standardized >  objective  instrument  : 
to  measure  attitudes  toTvard  cent  rail  j^sd  versus  local  control. 
AssiiTSie  that  tiortor,  on  i:hi::^  ir.etruTnent:  were  available  for  the  nation 
as  a  whoX^ivi -t^t^^"  blackout;  r<^i?;iori.  a-ici  the  region  outside  the,  blackout 
ar^^aV    He  fj-slected:  .05  per:  cent:  ran^on  sa::^ples  frori  the; citizens 
21  yearo  or  age  or;  oidar  living  In  thn  bl^^ckout  are^  and  from  ; 
thoBe  living  outs^ide  the  blackout  area.    He  adsilnistered  and 
scored  the;  ins tr uinerit  and  tested  the ;  sigut f  Ic snce  of  the  difference 
between  average  scores  for  the  tv?o  groups  /    He  found  a  marked  : 
dliffer^ence:    the  persons  living  in  the  blackout^^^^^z^^ 
opposed  to  Federal  control,    ne  thus  c^^ 

did  in  fact  affect /attitudes {toward; cen  tor  those, 

who  actually  experienced  the  blackout. 

IfHiat  Bia jor  type  of  inquiry  best  characterizes  the  design  of  this  study? 

■      Vex' po3trfacto'"research  ■;  ■ 

b.  /  trtje  expexitiiental  ixesearch;  ;  "/^  ^ 
■c*-:'  savnple;3urvey' research' 

d.    attitudinal  lnvf^nr.ory,  research 

Which  of  the  following  probletns  would  most  require  historical  research? 

a.    tr*?.iid3  in  patterns  of  financing  Georgia  public  schools 

b  •    current  concepts  of  "discipline-'*  of  public  high  school  principals 

in-' Georgia  ,  ■(/■■^V''v--'.:'^:' ■ . 

X.    effects  of  "social  acceptance''  on  tetxdencics  to  "withdraw" 

the  d<rv^elopmental  neede  of  high  school  youth  in  Clarke  Countyj,  ; 

\;Georgia..  , 

Wiica  one  of  the  folloT>xing  is  an  appropriate  step  i:o  take  in  the 
extexyial  criticisfti  of  an  c7-iginai  docuTflent  used  ii.  historical  research? 

a.  .  determlninf!;  tJbo  actually  wrote  the  docutnent  . 

b .  establishing  the;  author possible  motives  for  witing  the  document v 

c.  niaking  a  study  to  determine  the  author's  meanings, 
d-  analyzing  the  author's  poBsi.ble  bias. 
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18.    One  of  the  weaknesses  of  the  historical  naethod  is  that  the  researcher 
;has  available  as  data  ''selected  conceptions  of  reality  rather  than 
.      reality  itself All  of  the  following  are  implications  of  this  ' 
.weakness  SXCEPT 

a.  important  facets  of  h^s:-  ;   ..^.crr.:-:^:,^  may  never  be  decessilSs/ 
to  the  researcher. 

b.  the  descriptions  or  reality  available  to  th«  researcher  iriay 
be  inaccurate  or  in  error.  ^ 

c>     the  findings  of  historical  research  rnuRt  he  considered 
undependable 

d.     external  and  internal  criticisms  of  the  data  are  crucial 
to  the  success;  of  the  r^^^search . 

19..  ;  When  reading;. a  research:  repott  .  in  a  respectable  journal,  ve  should.  •• 

a.-  accept  the  concluEions  without  question, 

by  :  accept  the  conclusions  without  question  only  if  the  author 
is  a  wcll-.lcnown  authority, 

c.  :  accept  those  conclusions  that  agree  vith  cur  opinion; 

d.  reje<it  all  conclusions  if  the 

■"■argutaeut-«.'  ^\  ' 

e.  eva.luate  the  couclusions  according  to  the  correctness  of 
the  research  methodology.  ' 

;20.    The  acceptance  of  experiisentaticn  in  education  has  been,  according 
to  Campbell  and  Stanley, 

a.  increasing  steadily  over  the  past  six  decades.  ;  -  - 

b.  sporadic  overall  but  highest  in  the  1920 ^s  and "lowest  in 
■  the; ■mid-1930 *Sr'      ;;-  ,  '. 

.  c«     decreasing  steadily  over  the  past  six  decades . 

d.     at  a  constantly  high  level  over  the  past  four  decades. 

21.  ;   A  journal  In  which  appear  brief  summaries  of  the  substance  of ; 

■     "primary  research  completed; x^^  q£ 
education  over  a  given  period  of  time  ; (e.g . ,  three  years)  Is  the 

^'     ^J/j-AW  of  Educational  Research  . 
b  >     Educational  Al^tracts^.^ 

•  ^  *  :  Journal  of  Educational  Research;-  - ■  ■ 

^>    .Educational  Research  ^ujj:^^:ln  > 

22.  The  most  recent  data  regarding; dependable  knowledge  in  a  given 
educational  field  are  likely  to  be  found  in 
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^*  ^^^"-tcyclopedia  of  Educational  Research* 

b>  A.  E/R>A/ Handbook 

^  •  c.  Yearbooks  of  the  National  Society"'for  the  Study  of  Education, 

j/^"  d*  current  r"esearch  periodicals. 


23. 


In  which' one  of  the  following  would  one  find  a  crit  f 
.the  Iowa  Test  of  Basic  Skills? 


b.     Encyclopedia  of  Educational  Research 

d:.    ^Educational ;  and  £s]^^^^ 

e*  pj  Educational  Research  ^ 

-^e  follox^ing  research  resources  apply  to  items  24  -  27  below: 

1.  Handbook  of  Research  on  Teaching 

Encyclopedia  of  Educational  Research 

3.  Review  of  Educational  Research         .  ' 

^  *  .NEA  Journal  ,  "  ■ 

■  .  .  5.  Readers  Guide 

6«  Psychblogical  Abstracts 

7 .  Psychological  Bulletin 

8.  Ttest 

9..  Mfental  Measurements  Yearbook 

jEfl .  American  Educational  Research  Journal 

Im  nhlch  woiaiiia  you  look  to  f 
24^    A  briefi^sumtaary  of  the  research  on  classroom  organization: 

 ,,,^>.,,^,;2:..^,^,,:::.::c),.-..3,,_.:  :_d>:/  .8„.:::;.   ..e).,  9  

25w    To  obtain  a  critical  review  of  the  Lorge-Thomdike  Intelligence  Test 

■■■,a)r^-l-^-r-^y    3,         c)     7,         d)     9,         e)  10 

26*    A  geneiiraa  and  complete  suinmary  of  research  ^^^w^  last  thre% 

years  om  "Educational  and  Psychological  Tes 

a)    X/      ib)     2,         c)    3,         d)     6,         e)  9 
27.    A  high  qxiali:ty  publication  of  original  research : 


a)  :  3:,       !>)    6^         c)     7,         d)  10 


'11.     Si:aps  in  the  Proce^^s  of  Research 

28.  In  conducting  research,  Which, of  zhe  following  steps  should  generally 
be  tfikon  i'lrst?  ■ 

a.  collection  of  data 

b.  ^'OTT.pilation  of  a  bibliography  of  siniii^^^ 
:              Ci  fonniiXatior).  of  -a  workirig  hypothesis 

d.  c?>.refu:L  forn^ui/itlon  of  the  prbblanv  no  be  solved 

29.  A  hypothesis  is 

a.  s.^  general  f^-atieTteut  which  has  been  proyed. 

b.  a  general  Gtacenent  not  subject  to  proof, 
a  stateivie5:iit:  temporarily  accepted  as  true; 

d.  ,  a  stateiEant  considered  as  false  until  proved  true.  V 

30v  The;  lowest  level  ^  hicrc^rchy  of  certainty  of  scientific  knowledge 

is  represent 2d  by  ■  the  • 

a.  hypothesis . 

b  ^  '  thsory ,  ... 

.  c ,  • law; .  : 

,  -  d\  principle. 

21^    A.substrntive  or  rese^^rch  hypothesis 
a.     is  directly  testable. 

h.    has  r.o  be  t^ranslated  into  operational  and  experlsnental  term?. 

c.  MS  dependent  on  the  statisui^-ial  hypothesis . 

d.  predicts  how  the  analysis  of  quantitative  data  will  be 
resolved. ■ 

32.  The  first  step  In  the  empirical  tesUng  of  ^ 

a.  collecting;  relevant  data. 

b.  deducing  consequences  that  can  be  observed* 
,  c.     selecting  or  developing;  test-r  or  observational  systems  that 

t^ill' provide. data.  . 

33.  wien  a  general  pr^^ 

makes  the  deei.^ion  to  undertake  a  research  prcject  towards  its 
solution,  he  uKist  first,  identify 

.  ^  a.     the  studios  he  v^-iii  patcern  his  projec 

Q  :  the  tests  available  to  solve  t 

ERJC"  '^^'^-ables  inherent  in  the  proble^^^ 

ma^iii  the  inferance  pattern  relAted  to  the  problem* 
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34.    The  scientist  affords  the  facts  a  chance  to  prove  or  disprove 
something  by  hypothesizing  In  advanca. : 


a.  True 

b.  "False 


35,  Hypotheses  are  statements  about  relations  between  variables. 

.  a> .  True 
•  b,.;  .  -False' 

In  items  36-40  evaluate  the  adequacy  of  each  statement  as  a  scientific 
hypothesis  by  using  the  following  key;. 

a.  Adequate  hypftthesis 

b.  Inadequate,  less  than  two  variables 

c.  Inadequate,  no  statement  of  relationship 

d.  -    Inadequate,  not  testable 

36.  If  progranmed  te>rt  materials  are  used  to  teach  arithmetic  drill 
materials,  then  mastery  of  che  raaterials  (90%  or  better  on  a  critetion 
test)  will  follow. 


37.    If  subjects  vary  in  their  level  of  anxiety  (as  measured  by  the  M.A.S.^ 
•  then  they  will  differ  in  their  achievement  on  a  timed  spelling  test.  ' * 


V   Frustration  (produced  by  preventing  children  from  reaching  solutions 
to  problems)  produces  agression-  " 


39.    If  people  are  prejudiced,  then  they  identify  minority  group  member£ 
by  their  faces  more  readily  than  do  unprejudiced  people. 


40..    Group  therapy  (as  opposed  to  non-group  therapy)  will  result  in 
individuals  with  raore  adequate  value  systems. 
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41 


42, 


43. 


44. 


45. 


S?«  in    f  ^  study  began  with  this  statement :     "l-he  purpose  of 
this^study^is  to  determi.ne  the  differences  beU/een  'under  achieving' 
and    tior  ^alxy  achieving '  students  1«  the  seventh  and  eighth  gra2^^^ 
with,respcct^to  variables        personality,  aptitude /stufv  habits! 
and  itome  environn^:.n;..       Is  the  statement  a  hypothesis?:  " 

because  the  statement  is  not  posed  in  the  form  of  a 
,  question. 

b.  No,  becauGe  It  does  not  propose  a  relationship  to  be  tested. 
"    oraif?::ences!  ^  ^-^ing  of  the  significance 

;aSSd°prSle^f'°'''"'^  '=  r^^^^^  Characteristic  of  an  .. e,uately 

.    a.    It:  is  .based  .on  previous  research. 

b.     It  is  developed  from  clearly  defined  objectives .  : 
.  It  is  based  on  the  analysis  of  data  already  availabJe 
:    d.     It  3.eads  logically  to  the  listing  of  testable  hypothes^ 

otJ^^"^?"^^-^^^  ^?^^^^«l«at;ing  the^:^ 

In  Study  ia:    Has  the  author  explained  his  procedurea 

tn  such  a  way  that  .ill  enable  the  reader^o  ^  t^^y, 

a.  --,  criticize  „'"'■■■■.;' ■ 

b .  cotnprehe.nd 

c.  statisticaily  analyze 
d  .  r.-:peat  ,■' 

e.  deduce  consequences  of 

,  In  selecting  a  group  for  a  study,  the  initial  step  is  to  " 

a.    identify^all  the  units  which  comp(>se  the  total  ponulation 
o!™.?A*'^^  °^  •^^'^bers  to  help  provide  an  unbiased* 

provide  for  double-sampling  techniques  to  insure  a  represen- 
tative sample  for  study,  represen 

^'    ySti^i^l  ^^f  ""'^ber  ;item  and  coinpare  the  obtained  results 
witn  a  sasnple  of  every  even  numbered  item. 

All^of  the  following  are  dangers  to  be  x^atched  for  and  weaknesses 
ta  be  corrected  in  the  collection  and  analysis  of  data^ 


■     a,.  -  Are  ' the  data;':  adequate?  ■  ■  : 
5 V  i^»  *:he  rdata  substantiate  the 


46.     The  £statlstlcal  laethods  to  be  used  1       study  should  be  considered 

a,  before  the  hypotheses  have  been  formulate  : 

b.  as  the  study  is  being  designed V  : 

c.  after  the  data  have  been  collected. 

d,  after  the  data  have  been  tabulated. 

A7»    A  null  hypothesis  in  a  cotnparative  e>q>eriment  is 
always  false, 

b.  always  the  same  as  the  research  hypothesis  .    ^  "^^^ 

c.  a  tstatistical  hypothesis  that  assumes  that  the  subjects 
in  two  treatment  groups  x^ere  equal  before  treatment  began, 

d.  -a  statistical  hypothesis  that  assumes  there^^^^i^^^^^ 
,■;  among  the  effects  of  treatments . 

V    e.    a  statistical  hypbthesis  that^^^^^  no  difference 

among  the  effects  of  treatments, 

48,    Defining  "learn  more*'  by  a  score  on  the  Iowa  Test  of  Basic  Skills 

■  -.is •' a 

a.  measured  operation  definition. 

b.  experimental  operatiouar  definition. 

c.  constitated  definition. 

d.  syn.onyni  definition. 

Read  and  exataine  the  following  statement:;-    For  each  term  on  the  leftp 
select  the  phrase  on  the  right  (taken  froni  the  statement)  which 
correctly  iliuBtrates  each  tetin.    Each  phrasG^^^m^ 

"What  effects  do  socio-econotaic  background  factors  have  on  students 
ability  in  visual  perception  as  measured  bv  performance  in  the  Renshaw 
;.  Visual;'Aculty-Test?"  ■ 

A9*        dependent  variable  a)  socio-economic  background  factors 

answer 

50*         construct /intervening  variable    b)  ability  in  visual  perception 
answer 

51r        operational  definition  c)  performance  in  the  Renshaw  Visual 

answer  J..' ^"Acuity. -Test;', ' 

52.        independent  variable 
.  ■  ■  answer  ' " 


Ill,    Experivnental  Research 


53.  :  Accordirig  to  Kerlinger,  one  basic  purpose  of  research  design  is 

a.  to  provide  research  questions . 

b .  to  suggest  research  problems . 

c.  to  control,  variance. 

d.  to  control  independent- variables^.^ 

54.  The  main  technical  funct^^^^^^  is  to 

a*     control  extraneous  systematic  variance* 
c*    maximize  systematic  variance. 

dv;-  :Xb):-,and  V(c)'  •    „.  ' 

e*    aJ.l  of  the  above 

A  sociologist  surveyed,  bymeani?  of  a  mail  questionnaire,  the  attitudes 
of; persons  who  man         a  certain  group  of  hotels:  and  restaurants  as 
to  whether  they  would  accept  Indians  as  guests  or  customers.    He  then 
iirranged  for  an^^.  visit  these  hotels  and  restaurants, 

and  subsequently  learned  from  the  couple  which  establishments  had  \ 
actually  e^^  He  found  that  of  the  establishments  which  had  \ 

served  the  Indian  couple,  over  90  percent  had  previously  stated  they 
•^ould  not  serve  Indians . 

;   E  of  action  is  not  necessarily 

a  reliable  indicator  of  behavior. 

a.  This  conclusion  is  warranted  by  the  study. 

b.  >  This  conclusion  is  net  warranted  by  the  study. 

Surveys  measuring  expressed  attitudes  contribute  little  to  the 
understanding  of  what  people  will  do  in  everyday  practice. 

a.  TViis  conclusion  is  warranted  by  the  study/ 

.     b-    This  conclusion  is  not  warranted  by  the  study. 

-V.     Campbell  and  Stanley  introduce  McCall's  work  as 

a^    an  example  of  the  best  current  thought  in  experimental  design. 

b.  a  natural  follow-up  for  the  work  done  earlier  by  Fisher • 

c.  an  early  yet  insightful  statement  of  fundamental  design 
,     techniques  and  methodologies. 

d»    an  example  cf  heavy  reliance  on  designs  of  the  pre -experimental 
type..  ■., . 

58.    A  sample  is  considered  to  be  random  if  every  element  of  the  population 
has  an  equal  and  independent  chance  of  being  includeid  in  the  sample. 

:'-'a»:  ■  True'V' ■„: 
\bV  .';  False '  : 
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-Statistics  is  to  paraineter  ag 

a.    population  is  to  sample. 
:   b.    sample  is  to  population* 
c,    iTiean  is  to  variance* 
variance  is  to  mc? an, 

^^0»    I^hen  a  researcher  states  that  3\  result  is  significant,  hv^.  means 

a,  the  effect  is  a  practically  important  one. 

b'.  the  gain  scores  are  not  correlated. 

c.  the  result  is, unlikely  to  be  a  .chance  occurrence. 

d-  the  sample  values  are  different. 

61*    The  results  of  an  experiraent  are  said  to  be  confounded  when 

a.  they  contradict  previous  research  findings, 
they  are  contrary  to  vhat  was  predicted; 

c.  they  are  unexpected. 

d.  the  effects  of  the  variables  cannot  be  unambiguously  evaluated, 

e.  tlie  variables  interact  with  each  other. 

v,)^.    Which  of  these  terms  least  belong.^  with  the  others? 

a*    experimental  de$igT) 
randoniization 

c.  correlation 

d.  manipulation 
e»  control 

Of  the  following,  the  most  injpor tan t  determinant,  of  the: extent 

to  which,  the  .  results  of  a  research  study,  cart,  be  generalised  is  the 

a-    nature  of  the  sample  used  in  thf:  study, ^ 

b.  adequacy  of  the  instrutnents  used  to  collect  data* 

■   .    c.    degree  to  which  the  effects  of  extraneous  variables  have 
.  been  controlied „.  ' 

ease  with "which  the  research  design  can  be  replicated* 

The  mj or  function  of  the  control  group-  in       experiiuental  study 
is  to  measure  the  effect  of 

a.    differences  in  difficulty  of  pre-  and  post^-  tests, 
.external  factors  upon  the  dependent  variahlc^. 

c.  iriteraction  of  group  characceris tics  and  v?>:perimental^ 
treatments 

d.  adrainistratlon  of  a  pre-test  prior  to  applicaticn  of  the 
experimental  treatment. 
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Use  the  follov/ing  to  answer  questions  65-69, 

A  coach  in  a  large  high  school  thinks  that  ballet  training  will 
improve  the  batting  average  of  his  baseball  ceam.    He  decides  to 
have  half  the  .teav.\  take  six  weeks  of  ballet  training  before  the  base- 
ball season  begins  while  the  other  half  does  not  take  such  training. 
He  will  then  compare  the  season  batting  averages  of  group  A,  those 
with  ballet  training,  and  group  B,  those  without  ballet  training,  by 
comparing  the  wean  of  group.  A  with  the  taean  of  group  B. 

65.    This  study  would  be  classified  as 

a.    a  school  survey 

^'    1^"  §^  ^£5.5.,  °^  casual-comparative  study  . 

c.  a  correlational  study. 

d .  a  trend  study .  , 

e<    experimsntal  rese^irch- 


66. 


67. 


68< 


69, 
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Ib  order  to  test:  his  hypotliesia,  the  coach  should  have  which  group 
take  bapLlet  training? 

a.  the  best  players 

b.  jthe  poorest  players 

gi  randomly  selected  grcap  ' 
dv   those  vjho  volunteer 

e.    |good  players  who  show  poor  coordination 
The  conprol  group  would  be  those  vho 

a.  tbke  ballet  training. 

b.  |io  not  take  ballet  training - 

c.  fCry  out  for  the  basiiball  team. 

d.  jiave  high  batting  averages,  • 

e.  are  willing  to  cooperate  iri  the  experiment 

The  indjapendent  variable  is 

a.  ballet  training 

b.  flatting  average  ■ 

c,  iruns  batted  in 

d,  the  size  of \the  school 

e»    the  grades  players  make  in  ballet  school 

This  stjudy  would  be  classified 

a.  one  group  pre--test/post-test 

by  control  group  pre-*test/post-- test 

c.  control  group  po.^t--test  only 

d.  iSolomon  four  group  design 
e»  |cor relational  study. 


suppose  the  group  wlch  ballet  training  completes  the  season  with 
a.  higher  mean  batting  average,  than  the  group  without  ballet 
training  and  statistical  tests  indicate  that  this  difference 
IS  highly 'unlikely  to  be  a  function  of  chance.    Assuming  thaf 
ao  problems  of  internal  validity  are  present,  the  coach  should: 

a.  reject  the  null  hypothesis. 

b.  reject  the  experimental  hypothesis. 

c.  declare  the  experimental  hypothesis  is  proven. 

The  external  validity  question  in  this  study  vould  be: 

a.  Was  ballet  training  really  responsible  for  the  differences 
in  batting  average? 

b.  Was  the  coach  unethical  in  making  baseball  players  take 
ballet  training? 

c.  Is  ballet  training  worth  the  expense  it  involves? 

d.  Would  ballet  training  improve  the  batting  performance  of 
other  teams? 

Hairthorne  effect  would  be  most  likely  to  increase  the  scores  of 

a.  the  best  players, 

b.  the  poorest  players. 

c.  the  players  v/ith  ballet  training. 

d.  the  players  without  ballet  training. 

e.  all  players . 

One  of  the  main  reasons  for  XTOrking  with  samples  rather  than  populations 

is  that  — * — —  

a.  populations ,  being  finite,  cannot  be  discussed  in  terms  of 
.  ,  intervals.:'  ■ 

b.  the  experimenter,  because  of  practical  reasons,  rasrely  can 
locate  or  test  the  population. 

c.  statistics  are : appropriate  to  use  with  samples  but  not 
appropriate  to  use  with  populations. 

d.  population  measures  are  biased,  whereas  the  repeated  use  of 
samples  controls  bias. 

Which  of  the  following  precedures  would  yield  the  most  appropriate 
data  for  studying  the  relation  between  Intelligence  and  achievement? 

:  a.    administering  an  achievement  test  and  an  intelligence 
test  to  one  sample  of  subjects. 

b.  administering  an  achievement  test  to  one  sample  of  subjects 
and  an  intelligence  test  to  another  sample  of  subjects . 

c.  administering  an  aLchieveraent  test  to  a  sample  of  subjects, 
all  of  whom  have  an  I,Q.  of  100. 

d.  adrai'^istering  an  achievement  test  to  2  samples  of  subjects 
and  an  iritelligence  test  to  2  different  samples  of  subjects . 
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75.    The  entire  collection  of  elements  or  measurements  about  which  an 
inference  is  to  be  made  is 


a.    an  experimental  design, 
'  b.    an  interval  estimate. 
c»    a  population, 
a  sample* 

76-    Experimental  and  control  grmtps  are  frequently  equated  by  matching 
pairsr^of  subjects  on  one  ori^aore  variables  (e*g.,  fhtelligSmce) 
and  assigning;  one  meinber  or  each  matched  pair  to  the  exper^Snentai 
group  and  the  other  member  ro  the  control  gruup,    Iftiat  f acrior  is  the 
most  crucial  in  detemining  the  effectiveness  of  this  procBdure 
for  developing  equivalent  groups? 

a*    whether  the  total  group  of  subjects  is  a  random  sample 

from  the  population  of  interest 
b*    whether  equal  numbers  of  males  and  females  xve re  assigned 

randomly  to  treatment  or  control  groups 
c.    whether  the  matching  variable (s)  are  exclusively  and  highly 

related  to  the  dependent  variable  of  interest 
d*    whether  the  study  is  experimental  or  e^  £2Lt  facto 

77,    Parameter  is  to  population  as 

a.  statistic  is  to  sample. 

b.  sample  space  is  to  frame  of  reference, 

c.  statistic  is  to  probability/ 

d.  non-parametric  is  to  parametric, 
statistic  is  to  mea.surement; 


^78.    If  ot  and  g  are  characteristics  o£  population  P,  and  if  a  and  b 
are  estimates  of  a  and  3  on  the  basis  of  a  sample,  then 

a.  a  and  P  are  statistics  and,  a  and  b  are  parameters* 
b»    a  and  6  are  parameters  and ,  a  ai\d  b  are  statistics / 

c,  ct,  3,  a>  b,  are  statistics. 

d.  6,  a,  b,  are  parameters. 

79* :  Which  of  the  following  terms  best  fits  the  following  definition? 
"A  conclusion  about  a  population,  made  on  the  basis  of  what  has 
happened  in  a  sample  of  observations ^  the  conclusion  being  drawn 
with  a  calculable  risk  of  being  in  error"  isS  a  (n) 


a ,  population  parameter  * 

b,  random  selection, 
statistical  infereTice. 

d.    unbiased  estimate* 


-16- 


Suppose  you  wish  to  compare  experimentally  the  effects  of  two  drugs 
on  reaetlon  tinie.    You  randotnly  select  30  patients  from  each  of  two 
hospiMls.    You  adiijinister  treaxmcnt  1  to  the  patients  in  hospital  A 
and  treatment  2  to  the  patients  in  hospital  B.    You  then  test  each 
subject  individually  on  a  single  apparatus  for  measuring  reaction 
time,  'Compute  the  laeans  and  standard  deviations  for  the  two  groups, 
and  calculate  a  two  tailed  t  test  using  the  .10. level  of  significance. 

80-    SShat  source  of  error  is  most  likely  to  result  from  this  design? 

a.    Initial  differences  between  hospitals  could  obscure  differences 

between  the  effects  of  the  drugs . 
)>•    Using  only  one  testing  apparatus  prevents  generalisation. 

Comparing  two  treatments  yields  too  .little  information; 

instead^  each  should  have  been  cois^pared  with  a  control* 
d.    Tlie  effects  of  one  drug  may  interact  with  the  effects  of 

the  other. 

31.    Is  the  statistical  analysis  appropriate? 

a.  YeSy  the  t  test  will  yield  a  valid  estimate  of  the  statistical 
difference  between  the  test  results  for  these  samples* 

b.  Yes,  the  t  test  will  yield  the  information  necessary  for 
determining  x^hich  drug  is  superior. 

c.  No,  a  one  tailed  test  is  required* 

d.  No,  the  .10  level* of  significance  is  too  large. 

82.  Which  of  the  following  correctly  states  the  extent  of  pt^vrmlssible 
generalization  in  the  formation  of  a  statif^tical  inference?  The 
gencrali2ation  pertains  to  all  .individuals 

a*  possessing  characteristics  similar  to  those  used  in  the  study, 
b,    actually  used  in  the  study, 

suggested  by  a  carefully  developed  analogy. 
d\    from  the  population  sampled* 

83.  Characteristics  of  persons  or  things  which  can  assume  different 
values  are  called 

a.  distributions. 

b.  observations* 
,       c,  data. 

d.    variables , 

;84.    In  an  experiment  to  Investigate  the  extent:  to  which  learning  changes 
systematically  with  increasing  age,  the  independent  variable  would  be 

a^  length  of  the  list  to  be  learned. 

b*  age  levels  of  those  tested, 

rrnr  c.  change  from  trial  and  error  to  insightful  learning - 

fcHJC  :  number  of  error?;  made  on  learning  trials. 
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S5.    Therifaandedness  of  a  person  is  a  (an) 

;a.  continuous  variiable. 

b.  discrete  variabis. 

c.  Independent  variable . 

d.  dependent  variable. 
^.  Intervening  variiable. 

m.    In  an  experiment "to  determine  the  affect  of  the  tempo  of  music  on 

workers  productivity  in  a  factory  setting,  the  dependent  variable 
18  tne 

.a.  tempo  of  the  aiusic» 

b.  factory  setting. 

c.  nuadjer  of  workers, 

d.  workers*  productivity. 

87.    Which  term  least  belongs  with  the  others? 

a.  dependent 

b.  ordinal 

c.  response 

d.  criterion 


88. 


89, 


A  dependent  variable  is  generally  a  classification  variable  which 
nay  be  manipulated  or  controlled  in  an  experiment. 


a.  True 

b.  False 


The  dependent  variable  Is  the  treatment  variable  which  tha  investigator 
varies  for  purposes  of  controlling  the  results.  ts«or 


"a.  ,  'Truev, 
b.  False 

90*  Students  taught  first  aid  by  programmed  Instruction  will  achieve 
at  a  higher  level  than  those  taught  first  aid  by  the  traditional 
method.    In  the  above  hypothesis  the  indapendent  variable  is 

a.  students. 

b.  level  of  achievement. 

c.  prograBBjed  instruction. 

d.  method  of  Instruction. 

e.  first  aid  test  scores. 
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E  plans  an  experiment  in  ^^nich  60  children  are  to  be  randomly  assigned 
to  four  treatment  groups:    extrinsic  reward  (candy)  and  extrinsic 
punishment  (loss  of  recess) ;  extrinsic  reward  only;  extrinsic  punishment 
only;  and  neither  extrinsic  reward  nor  punishment.    Before  the  study 
begins,  E  gives  an  IQ  test  to  all  the  pupils  to  use  in  the  statistical 
analysis,  but  not  to  be  used  in  assigning  pupils  to  groups.    To  determine 
the  effect  of  the  treatments a  geography  test  is  given. 

91.  In  this  experiment,  the  assigned  independent  variable  is 

a.  treatment* 

b.  IQ. 

c.  geography  test  score. 

There  is  none  in  this  experiment • 

92.  In  this  experiment,  the  active  independent  variable  is 

a.  treatment. 

b.  IQ. 

c.  geography  test  score. 

d.  There  is  none  in  this  experiment. 

E  plans  an  experiment  to  determine  if  use  of  certain  specified  amounts 
of  a  drug  will  increase  the  IQ  scores  differentially  for  male  and  female 
pupils.    He  uses  fifth  graders  for  the  study. 

»        .    ^  t"  ■ 

93.  In  this  experiment,  IQ  serves  as 

a*  an  active  independent  variable. 

b.  an  assigned  independent  variable. 

c.  a  dependent  variable. 

d.  Both  A  and  B  are  correct. 

94.  In  this  experiment,  sex  serves  as 

a.  an  active  independent  variable* 

b.  an  assigned  independent  variable. 

c.  a  dependent  variable. 

d.  None  of  the  above  are  correct. 

95.  In  this  experiment V  the  drug  serves  as 

a.  an  active  independent  variable. 

b.  as  assigned  independent  variable. 

c .  a  dependent  variable. 

d.  None  of  the  above  are  correct. 

96.  The  experimenter  usually  has  as  his  objective  the 

a.    minlmizittg  treatment  or  experimental  variance, 
9^  b.    maximizing  extraneous  variance. 

c»    minimizing  error  variance. 
None  of  the  above. 
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TheEnglisK  department  of  the  Sunbury  Junior  High  School  is  concerned 
with  improving  the  reading  speed  and  comprehension  of  7th  graders 
They  have  decided  to  conduct  a  training  sxperinent  uCilialng  the 
Tachitron  and  appropriate  sets  of  training  cards.    m±le  developing 
the  design  for  the  study,  several  important,  considerations  were  raised. 
First,  it  appeared  crucial  to  be=able  to  determine  whether  or  not 
improvements  in  reading  skill  as  noted  through  scores  on  a  reading 
achievement  test  would  in  fact  be  due  to  the  training.    Second,  it  was 
also  realized  that  in  order  to  avoid  mistakes  and  inconsistencies  in 
grading,  it  would  be  important  to  develop  an  objective  and  accurate 
method  of^scoring  the  test.    There  was  also  concern  about  the  effects 
of  other  factors  on  the  results  of  the  study.    It  was  noted,  first." 
that  many  of  the  students  came  from  disadvantaged  homes  and  had  limited 
language  experiences.    Other  students  had  come  from  a  grade  school 
system  that  employed  a  special  training  program  in  reading.  The 
intelUgence  range  of  the  students  was  quite  wide  and  might  also  be 
an  Important  factor. 


found  in 


97.  In  the  above  study,  experimental  variance  would  be 

a.  Intelligence  test  scores. 

b.  scores  on  the  reading  achievement  test. 

c.  the  research  design  of  the  study. 

d.  the  method  of  scoring. 

98.  Which  of  the  following  constitutes  a  type  of  error  variance  controlled 
in  this  study? 

a.  socio-economic  levels 

b.  the  initial  low  levi^l  of  reading  ability 

^*  mistakes  and  inconsistencies  in  scoring  the  achievement  test 

d.  the  training  method 

99.  An  example  of  An  extraneous  variable  in  the  study  would  be 

a.  previous  participation  in  a  reading  training  program, 

b*  scores  on  the  reading  achievement  .teat. 

c»  the  training  method. 

d.  mistakes  and^ inconsistencies  in  scoring. 

100.    "A  'true ^  experimental  design  must  incorporate  pre-'test."    (In  the ^ 
Campbell  &  Stanley  sense)    This  statement  is 

a.  true. 
■-■ .  b.  false. 

c.  usually,  but  not  always,  true. 

d.  usually,  but  not  always,  false. 
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101 •    When  a  ipre-test  and  post-test  are  used  on  all  subjects.  In  the 
experlxnental  and  control  groups 

a.  the  pre-test  may  influence  post-'test  results. 

b.  the  E  knows  whether  it  is  reasonable  to  assume  that  both 
groups  began  at  a  similar  point  or  not. 

c.  there  may  be  interaction  between  pre-test  and  treatment 
effects. 

d.  a  and  b  are  both  true,  , 

e.  a,  b,  and  c  are  all  true. 

If  Method  A  works  better  with  bright  (intelligent)  students  and  Method  B 
with  dull  .students  (as  measured  by  the  ABC  test,)  and  the  pattern  is 
even  more  pronounced  for  boys  thfin  girls,  we  know:  o 


Indicate  True  or  False  for  items  102-107  A 
102.    Girls  achieved  better  than  boys. 


a.  True  ir. 

b.  False 

103.  The  overall  girl  and  boy  means  differed  significantly- 

b.  False 

104.  There  was  a  sex-by*-intelligence  Interaction. 

a.  True  ^ 

b.  False 

105*    There  was  a  method-by-intelligence  interaction. 

a.  True 

b.  False 

106.  There    was  a  method-by-'intelligence-byseic  interaction. 

.   ■  .  a:.  ■;  True;  ■. 
b.  False 

107.  The  overall  means  for  Method  A  and  B  differed  significantly* 

a.  True  ;.'  .  ■  ■ 

b.  False 

108 .  In  the  above  factorial  design ,  how  many  factors  are  there? 

a)    0,     b)    1,      c)    2,     d)    3,     e)    6  or  more 

109.  How  many  dependent  variables  are  ther^? 

a)    0 ,      b)    1,      c)    2,      d)    3,      e)    4  or  more 


-21- 


110.      Which  type  of  sampling  is  used  if  the  population  Is  divided  into 
groups  by  . certain  common  characteristics  and  random  sampling  is 
done  from  each  e^roim? 


done  from  each  group? 

a.  cluster 

b.  systematic 

c.  stratified 
d»  proportional 
e .  double 


111.  A  researcher  draws  a  sample  by  selecting  every  lOth  name  In  a 
telephone  directory.    \^±ch  of  the  following  sampling  types  has 
he  used? 

a.  simple  random, 

b.  double 

c.  stratified 

d.  systematic 

112.  If  50  students  from  a  group  of  200  7th  tenners  were  to  be  studied 
by  the  Rorschach  technique,  the  sample  (50)  would  be  considered 
random  if 

a*    there  were  the  same  nuii>ber  of  boys  and  girls  in  the  group 
of  50. 

b.  every  boy  and  girl  in  the  7th  term  had  an  equal  opportunity 
to  be  selected. 

c.  two  students  were  selected  whose  names  began  with  each  of 
the  25  letters  of  the  alphabet  except  X. 

d.  every  IQ  level  (ex:    80-85,85-90,90-95,  etc.)  was  equally 
represented  by  the  50  students. 

113 ,  Selecting  the  children  to  be  involved  in  an  experiment  by  randomly 
identifying  the  schools  to  be  involved  represents 

a.  stratified  sampling. 

b.  double  s^jnp ling. 

c.  systematic  sampling* 

d.  cluster  sampling. 

114,  Two  characteristics  of  a  random  sample  are:     (1)  independence  of 
selections,  (2)  equal  probabilities  of  selection. 

115 •      If  a  sample  is  large  and  randomly  drawn,  then  it  is  quite  likely 
it: is  representative  of  the  population. 

a.  ■True'  '• 
b*  False 


erJc 
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116.      Random  sampling  guards  against  knoxm  and  unknown  biases. 

a.  True 

False 

117*  Separating  a  . group  of  possible  subjects  into  categories  (e,g,  males 
and  females)  before  selecting  subjects  (by  use  of  a  table  of  random 
numbers)  to  receive  a  particular  treatment  is  known  as  drawing  a 

a,  stratified  random  population, 

b*  simple  random  sample. 

c.  unbiased  random  sample • 

d.  stratified  random  sample. 

—3.      When  a  sample  has  been  drawn  at  random,  we  can  still  only  make 

probabilistic  statements  about  its  relationship  to  the  population 
from  which  it  was  drawn c 


119.      Selecting  a  sample  at  random  means  that 

a.    the  sample  drawn  will  definitely  be  representative, 
.  .  b.    every  member  of  the  group  drawn  from  has  an  equal  and 
independent  opportunity  of  being  selected, 
c.    Both  A  and  B  are  correct, 
'd.    Neither  A  nor  B  is  correct. 

'''^lev     A  sample  selected  to  assure  that  certain  subgroups  in  the  populatio 
will  be  represented  in  the  sample  in  proportion  to  their  numbers 
in  the  population  itself  is  called  a 

a.  random  sample.  ^ 

b.  stratified  sample. 

c.  pilot  sample. 

d.  heterogeneous  sample, 

^^x.      The  most  efficient  means  of  ruling  out  factors  such  as  maturational 
level,  ability ,  age,  etc. ,  as  rival  explanations  of  an  observed 
post-test  difference  between  an  experimental  and  a  control  group  is 


True 
False 


careful  matching  of  the  experimental  and  control  groups  on 
all  possible  factors  (e^g. ,  sex,  IQ^*  and  achievement)  . 
a  time-series  quasi-'experimental  design, 
random  assignment  of  subjeots  to  the  treatment  groups, 
the  administration  of  a  prerrtest. 
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122.  An  Investigator  wishes  to  evaluate  the  influence  of  three  methods 
of  teaching  reading:  the  basal  reader  method,  the  I.T^A.  method, 
and  the  individualized  reading  instruction  method.  In  setting  up 
his  groups  for  study,  he  should  be  sure  to 

a.  assign  the  children  randomly  to  classes. 

b.  assign  teachers  randomly  to  classes. 

c.  assign  the  method  of  reading  instruction  randomly  to  classes. 

d.  all  of  the  above. 

e.  none  of  the  above. 

123.  In  a  large  high  school  the  25  students  who  scored  highest  in  an 
English  pre-t est  were  placed  in  a  special  class.    At  the  end  of 
one  semester  they  were  given  another  English  achievement  test. 
The  mean  grade  level  equivalents  were  compared  for  the  two  tests. 
This  is  an  example  of 

a.  one  group  pre-test /post-test  design. 

b*  randomized  control-group  pre-test/post^-test  design. 

c*  randomized  Solomon  four  group  design. 

d.  randomized  control-group  post-test  only  design 

e.  factorial  design, 

124.  The  ''halo"  effect  in  rating  refers  to 

a.  influence  of  one  rater  upon  another 

b.  tendency  to  rate  a  person  higher  when  you  know  him  better. 

c.  spread  of  a  general  impression  of  a  person  to  the  rating 
o  of  specific  characteristics. 

d.  tendency  to  make  ratings  too  high. 

e.  tendency  not  to  be  too  hard  in  rating  people  one  doesn*t  like. 

125.  Subjects  performing  well  merely  because  they  are  being  observed 
(and  not  necessarily  because  of  any  effect  of  treatment)  are 
considered  to  be  under  the  influence  of 

a.  the  Hawthorne  effect. 

b.  the  novelty  effect. 

c.  the  halo  effect. 

d.  none  of  the  above. 

126.  The  internal  validity  of  an  experimental  design  is  concerned  with 
the  question i 

•  a.    to  what  extent  can  the  experimental  findings  be  generalized? 
b.    idid  the  independent  variable  really  produce  a  charge  In  the 

dependent  variable  ? 
c*    how  representatiye  is  the  setting  selected  for  the  experiment? 
d.    will  the. findings  provide  information  about  situations  in 
• which  variations  of  the  independent  variable  are  present? 
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127 •    When  the  results  of  an  experiment  are  true  only  under  certain  very 

special  conditions  and  do  not  generalize  far  beyond  these  particular 
experimental  conditions,  the  experiment  lis  said  to  have 

a.     low  internal  validity. 

low  external  validity, 
c.    high  external  validity. 

128.    Which  of  the  following  is  an  internal  validity  question  in  experimental 
research? 

a.  '  Has  maturation  influenced  the  dependent  variable? 

b.  Was  the  sample  representative  of  the  population? 

c.  Has  the  hypothesis  been  stated  in  operational  form? 

d.  How  widely  CAn  the  results  of  the  experiment  be  generalized? 

e.  .  Are  the  potential  results  of  the  study  worth  the  cost? 

129*    Which  term  least  belongs  with  the  others? 

a.  internal  validity 

b.  external  validity 

c.  generalizability 

d.  representativeness 

130.  "To  what  populations,  settings,  treatment  variables,  and  measurement 
variables  can  this  effect  be  generalized?**  might  most  appropriately 
be  asked  In  relation  to  the  concept  of 

a.  criterion  validity. 

b.  Internal  validity, 
c*  external  validity, 
d.  construct  validity. 

131.  "Did  In  fact  the  experimental  treatments  make  a  difference  in  this 
specific  experimental  instance?"might  most  appropriately  be  asked  in 
relation  to  the  concept  of 

a»  criterion  validity, 

b.  Internal  validity* 

c»  external  validity* 

d,  construct  validity. 

132.  In  evaluating  a  research  project,  what  are  the  two  most  important 
■  criteria?.. 

a.  Internal  validity  and  external  validity, 

b.  representativeness  and  generalizability. 

c.  practical  significance  and  statistical  significance. 

d.  internal  consistiahcy  and  reproducibility* 
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133.    A  pre-test:  of  achievement  is  given  to  an  unselected  group  of  ^0 

students  five  minutes  prior  to  an  hour's  instruction  in  addition. 

The  post-test  that  immediately  followed  the  instruction  showed 

a  10  point  gain  from  the. pre~test •    It  Is  concluded  that  the  ins true tio: 

was  effective  in  producing  a  gain.    Which  one  of  the  following 

sources  of  invalidity  could  also  explain  the  pre-test  to  post-test 

gain? 

a.  statistical  regression 

b.  interaction  of  selection  and  maturation 

c.  instrumentation 

d.  testing 

134*    Professor  "X"  graded  initial  essay  exams  for  his  class,  then  graded 
the  final  essays  to  see  if  improvement  had  occurred.    Which  of  these 
least  jeopardizes  internal  validity? 

a.  history 

b.  maturation 

c.  testing 

d.  instrumentation 
e*  regression 

135.    Which  i»ould  probably  be  the  most  serious  threat  to  internal  validity? 

a.  history 

b.  maturation 

c.  testing 

d.  instrumentation 

e.  regression 

136*    A  threat  to : internal  validity  that  may  occur  from  processes  within 
respondents  operating  as  a  function  of  the  passage  of  time  per  se 
(and  not  specific  to  particular  events)  is  known  as 

a,  maturation 

b»  instrumentation 

c .  regression 

d.  reactivity 

137*    When  groups  selected  for  their  extreme  scores  on  a  pre-test  are  used 
in  an  experiment,  a  likely  source  of  invalidity  in  some  designs  is 

: a.  ■■.selection 
..  b*  maturation 
c»  regression 

d,    interaction  of  selection  and  treatment 
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13.8.:   In  expGriinarital  design,  when  comparisons  are  made  of  groups  which 
have  been  selected  on  the  basis  of  their  extreme  scores,  the  post- 
test  means  of  the  groups  tend  to  move  toward  the  means  of  the  entire 
population  from  which  the  extreme  groups  were  selected.  This 
factor  which  jeopardizes  validity  of  the  findings  is  termed 

a.  differential  selection 

b.  interaction  of  selection  and  X, 

c.  experimental  mortality 

d.  statistical  regression 

139.    Which  of  tha  following  is  a  factor  jeopardizing  the  external 
validity  of  an  experiment? 

a.    statistical  regression 

b»    interaction  of  selection  biases  and  the  experimental  variable 
c.  instrumentation 
.  d,  history 

140/    Children  having  attended  nursery  school  were  coanared  with  those 
having  no  such  experience  on  a  readiness  test  at  the  beginning  of 
Kindergarten  and  were  found  to  be  superior.    The  research  attributed 
this  superiority  to  the  children's  nursery-school  experience- 
The  greatest  threat  to  internal  validity  is 

a.  history 

b •  maturation 

c.  testing 

d.  selection 

e.  .    eMperimental  mortality 

141.  Pupils  in  the  remedial  reading  program  increased  in  problem  solving 
ability  during ; the  course  of  the  year.    Attributing  the  cause  to 
the  change  of  the  program  overlooks  several  alternative  factors , 
the  most  significant  of  which  is  probably 

a.  selection 

b.  testing 

'    cy    experimental  mortality 
d.  regression 

142.  Students  with  low,  average  and  high  initial  self -esteem  were 
cc7:r>.?3eled.    The  low  group  gained  most  over  2  months  ♦  Assuming 
the  counseling  was  more  effective  with  the  low  group  falls  to 
consider  (greatest  threat) 

a.  history 

b,  maturation 
c*  testing 

d»  Instrumentation 
e*  regression 
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IV.    Methods  of  Data  Collection 

143.  Annual  achievement  tests  which  are  parts  of  the  school  testing 
program,  illness  records,  daily  assignaaents  and  other  routine 

characteristics  and  activities  of  the  school  are  considered  to 

■•  ■  be  ■ 

a.  reactive  measures 

b.  nonreactive  measures 

144 .  Which  of  the  following  is  most  likely  to  provide  the  best  description 
of  the  objectives  of  instruction  in  a  school  system? 

a.  teacher  ratings  of  the  students 

b.  standardized  aptitude  tests 

c.  teacher-made  achievement  tests 

d.  standardized  achievement  tests 

145 .  In  which  way  are  teacher-made  teats  superior  to  standardized 

■  'tests?; 

a.  They  are  more  reliable  for  evaluating  differences  among 
very  poor  and  very  good  students. 

b.  They  provide  more  valid  measures  of  the  teachers'  specific 

;■  ■■objectives  ,,-'^::'-';vV;K:^^ 

c.  They  provide  a  better  measure  of  the.  student's  grasp  of 
Important  facts  and  principles. 

d.  They  are  simpler  to  administer  and  score. 

146.  Which  of  the  following  is  most  characteristic  of  skilled  as  opposed 
to  unskilled  observers  in  research? 

a.  making  generalized  descriptions  or  evaluations  rather  than 
noting  specific  incidents 

b.  interpreting  behavior  on  the  basis  of  limited  data  rather 
than  waiting  for  confirmation  by  subsequent  incidents 

c.  giving  the  child's  behavior  rather  than  the  personal  reaction 
of  the  observer' 

d.  recording  primarily  dramatic  or  negative  Incidents 

147.  Which  of  the  following  is  essential  to  skilled  observation  as 
a  research  technique  in  studying  a  pupil's  behavior? 

a.    recording  primarily  dramatic  behavior 
,     b.    recording  primarily  the  "whys"  of  the  behavior  observed 

c.  recording  primarily  the  "whats"  of  the  behavior 

d.  using  data  obtained  as  bases  for  generalizations  concerning 
the  beWivlors  of  other  pupils 
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■S.    According  to  Kerlinger,  sampling  of  behavior  by  the  observation 
method  can  be  considered  ais  consisting  of  two  aspects: 

a.  person  sampling  and  event  sampling 

b .  pe  rson  sampling  and  time  sampling 

c.  event  sampling  and  time  sampling 

d.  .    criterion  sampling  and  person  sampling 

149.    Is  it  important  to  follow  up  those  persons  in  a  survey-sample  vho 
have  failed  to  return  questionnaires  mailed  to  them?     (Assume  that 
the  direct-mail  method  is  appropriate.) 

a.  No,  because  a  certain  loss  must  be  expected. 

b.  No/ because  such  delayed  returns  A^ould  be  of  doubtful  value. 

c.  Yes,  because  the  sample  at  hand  may  be  biased, 

dn    Yes,  because  the  size  of  sample  should  be  as  large  as  possible 

^^0.    Interviews  and  questionnaires  as  a  data  collection  method 

a,  are  more  effective  than  observational  techniques. 

b»  reveal  only  information  the  subject  is  willing  to  report. 

c.  cannot  be  considered  to  have  validity. 

d.  provide  no  information  about  past  behavior. 


3l.    Birect'-mail  questionnaires  have  whi  these  factors 

most  seriously  reducing  the  validity  of  their  "findings"? 


a.  history 

b.  regression 

c.  selection 

d.  Instrumentation 

e.  reactivev effects 

.w^;    X^hlch  of  the  following  is  usually  the  most  serious  limitation  In 
the  use  of  the  mailed  questionnaire  in  descriptive  research? 

a.  defining  a  population 

b.  selecting  a  sample 

c.  obtaining  responses 

d.  analyzing  the  data 

153,    Which  one  is  not  a  characteristic  of  rating  scales? 

a.  difficult  to  construct 

b.  subject  to  bias 

c.  easy  to  administer 

d.  measures  remembered  behavior 
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APPENDIX  B 


PROCESS  EVALUATION 


ERIC 


BUREAU  OF  EDUCATIONAL  RESEARCH  AND  TESTING  SERVICES 

p.  O.  BOX  Q 
DURHAM,  NEW  HAMPSHIRE  03824 


UNIVERSITY  OF  NEW  HAMPSHIRE 
Department  of  Education 


December  13,  1971 


Telephone:  862-1 8= 
862-18 
862.1  a 


Dr.  John  C,  Egermeier 
Acti  ng  Ch  ief 

Research  Training  Branch 
National  Center  for  Educational 

Research  and  Development 
Department  of  Healthy  Education 

and  We  I  fare 
Office  of  Education 
Washung+on,  D.C*  20202 

Dear  John: 


We  have  finally  received  permission  from  all  the  consultants 
to  publish  the  Final  Report  on  the  New  Hampshire  Summer  Research  Train- 
ing I nsti tutes. ; 

You  will  notice  that  I  have  enclosed  a  copy  of  the  Table  of 
Contents  of  the , Supp I ementary  Appendix  and  identified  the  items  which 
should  not  be  included  in  ERIC.    Additionally,  CIRCE  has  requested 
that  their  Final  Report  on  the  1970  Institute  be  submitted  to  ERIC  in 
total.    Accordingly,  the  CIRCE  final  Evaluation  Report  should  be 
included  in  the  section  titled  Process  Evaluation.     I  have  enclosed 
a  copy  of  their  Report. 


ERIC 


to  ca I  I  me , 


Should  you  have  any  further  questions,  please  feel  free 


Warm  regards, 

Maurice  0  I i v  ier 
Assistant  Director 


MO:cp 
Enclosures 


EVALUATION  REPORT 


UNIVERSITY  OF  NEW  HAMPSHIRE  SUMMER  INSTITUTE: 
PLANNING  FOR  EDUCATIONAL  DECISION  liAKING  IN  THE  70s 

July  6  -  August  14,  1970 


James  L.  Wardrop 
Terry  Denny 

Center  for  Instructional  Research  and  Curriculum  Evaluation 

Univers  ity  of  I iiiaois  ac  Urbana-Chairipaign 

October  1970  ' 


Time  it  was. 

And  what  a  time  it  was. 

It  was ... 

A  time  of  innocence , 
A  time  of  confidences. 
Long  ago  —  it  must  be..,, 
I  have  a  photograph. 
Preserve  your  memories; 
They*re  all  chat's  Left  you. 

--Paul  Simon,   "Bookends  Theme 


EVALUATION  REPORT, 

UNIVERSITY  OF  I^W  HA>IPSHIRE  SU^^^fER  IN^ 
PLANNING  FOR  EDUCATIONAL  DECISION  MAKING  IN  THE  70s 

,   .  ;     July '6  -  August  14,  1970 

:  It  all  bisgan  with  the  submission  of  : a  proposal  by  Gilbert  Austin  of 
l',:,  .    :  the  Unive^  New  Hampshire  to  the  U.  S.  Office  of  Education  for  a  summer 

■    institute  whose  topic  was  to  be  ^^lanning  for  Educational  Decision  Making  in 
tne  70s.  *V    At  Austin's  request,  members  of  the  Center  for  Instructional  Resea^rch 

■  ;    and  Curriculuni  Evaluation  at  the  University  of  Illinois  agreed  to  undertake  an 

"external  evaluation"  of  the  Institute ,     This  is  a  report  of  that  evaluation. 

•  As  frequently  happen;     no tification  of  funding  for  the  lxiStiV  :\ttV  was  , 
.    ; l^te  in  coming.     Thus,   the  time  available  for  recruitment  of  participants  was 
.short.     Additionally,  by  the  time  the  Institute  was  fundea,  Austin  was  no  longer 
at  the  University  of  New  Hampshire .     Albert  Elwell  stepped  in  as  Institute 
Director.     This  change  in  directors  after  the  structure  of  the  Institute  had 
:      ^  ::been  established  probably   (the  temptation  is  to  say  "undoubtedly ")  accounts  for 
many  of  the  difficulties  encountered  later.     Elwell ,  Everett  Barnes  (administra- 
tive assistant) ,  and  Maurice  Olivier  (coordinater  of  program  p  now 
i  .    :faccd  with  the  following  problem:     to  recruit,  accordinq  to  critGri.'^.  thev^  l:::.d 

not  selected,  thirty  qualified  participants   for  an  Institute  they  had  not  designed, 
within  a  period  of  about  six  weeks  from  the  time  funding  was  awarded  until  the  . 
.Institute  was  to  begin. 
c:  =  .  The  remainder  of  this  report  represents  an  attempt  (1)   to  document  the 

goods  and  bads,   the  rights  and  wrongs  of  the  New  Hampshire  Institute  and  (2)  to 
:  ■  abstract  from  these  findings  some  general  impressions  about  the  place  of  institutes 
i  world  of  professional  education  and  the  process  of  evaluation  of  sucf^ 

institutes.     The  report  is  organized  in  terms  of  four  aspects  of  the  Institute: 

■  staff consultants ,  content ,  and  part ic ipants.     It  is  an  attempt  to  characterize 


each  of  these  aspects  from  the  point  of  view  of  the  participants  in.  consultants 
to,  and  evaluators  of  the  Institute. 

EXTERNAL  EV  ALU  AT  ION 
Different  people  at  different  times  have  seen  CIRCE's  external  evalua- 
.  tipn;  task  within  the  Umi  di  project  developer.  Gil  Austin,  ] 

originally  intended  that  CIRCE  would  produce  a  hard-nosed  report  of  the  work  of 
thejnstitute  to  complement  the  internal  hard-nosed  evaluation  to  be  conducted 
;    by  the  staff .CIRCE  originally  intended  it  as  an  opportunity  to  integrate  several 
monitoring  and  data-collection  techniques  (unobtrusive  measures ,  part icipant 
observers,  impressionistic  judgn^ents)   thought  to  be  useful  in  broadening  the 
view  held  by  Austin.  j 

■   .    :  .  i^uch  views  proved  to  be  in  contrast        that  presented  during  the  suimer 
by  a  ^SOE  official.     He ^ 

the  Institute.     As  he  saw  it.   it  was  CIRCE's  rol.:  to  evaluate^^^:t^    utility  of  product; 
packages   ('•exportables'')  yielded  by  the  Institute   for  a  second  generation  of 
institutes  to  be  conducted  in  the  surnmer  of  1971.  i    ,      ^  ,  . 

Subsequent  project  director  AlVrciwell  valLied  CTnr'^-  n.-i.^^ 
ability  to  provide  him  with  .aperiodic  evaluative,  counsel  and  (process )   infom^ation  " 
during  the  -"^^""-3  well  as  for  providing  hin,  with  a  s  ^ 
In  retrospect  it  appears  that  all  these  roles  were  played  by  CIRCE:     one  well .  . 
another  badly--none  in  a  manner  to  war  rant  rave  reviews .        .        '  ; 

carrying  out  these  evaluative  functions^,^^^^^^^ 
infonnation-gathering  techniques.     For  the  process  evaluation,  we  relied  on^ two 
major  sources  of  information:     several  one-to-thrcc-day  site  visits  by  the  eval- 
uators to  the  Institute,  during  which  we  observed,  lis^t^^^^^^^ 
;  Weekly  Part icipant  Opinionnairo ,  by  means  of  which  we  obtained  systematic  feedback 
1 
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•from  the  participants  about  their  reactions  to  each  week's  activities.     (A  copy 
of  this  Opinionnaire  and  all  other  instruments  used  in  our  evaluation  will  be 
found  in  the  Appendix  to  this  report .)     An  early  expectat ion  was  that  the  partic 
ipant  observers  would  provide  the  evaluators  with  a  third  kind  of  infonnation 
through  a  series  of  regular  reports)^  This  expectation  proved  wrong .  ■     ^  ^  - 
(\For  a  sunmiativG  evaluation,  we  made  use  of  some  of  the  ••process" 

inforn^tion  obtained  during  the  course  of  the  Institute  but  incorpora 
data  gathered  by  means  of  (1)  an  Orientation  to  Evaluation  scale  given  at  the 
beginning  and  again  at  the  end  of  the  Institute,   (2)   "self-interviews"  from  the 
instructional  staff  (consultants),  and  . (3)  a  Sun^ary  Opinionnaire .     Copies  of  " 
^    these  instruments  are  included  in  the  Appendix..    (On  the  Summary  Opinionnaire . 
frequencies  of  responses  to  the  various  alternatives  have  been  provided I  " 

,  V  ADhnwiSTR.'^TIVE  STAFF-    ■     ■'^  j: 

f ;  ■  ■"   "  ■     ■         .      ■"  '■ 

\^In  any  educational  endeavor,  some  things  are  done  right,  some  wrong. 

The  former  tend  to  get  over!  nn^c^     fv„  i„..^.,.  ,      ^  . 

^  °°"^^>  '^"^  overempna-si>ecl.  -  Although  we  have 

tried  in  this   report  not  to  overlook  the  things   that  ..ere  done  ri3h^  ^nd  nr,t  to 

overemphasize  the  things  that  were  done  wrong,  it:  is  unlikely  that  we  have  been,  " 
completely  successful.  •  . 

The  InstU        staff-Elwell.  Barnes.  Olivier,  and  others  from  the 
.University  of  New  Hampshire--did  an  excell^ 

participants:  in  a  very  short  time.     The  "General  Information  Handbook."  desisned  : 
by  Elaine  Gardner.   Institute  secretary  and  technical  staff  assistant  and  issued 
to  all  participants  and  consultants .  was.  also  a::  very  thoughtfuL:  and  usefuI^  docu-  ; 
ment  insofar  as  it  provided  general  information  to  the  .participants  and  instruc-  ; 
tional  consultants  about  the  Institute  and  the  local  area. 
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:     Throughout  -ha  Institute  these  administrative  staff  members  showed 
themselves  to  be  energetic ,  patient ,' and  attentive  to  and  concerned  about  the  ' 
needs  and  desires  of  th-    Institute  participants.     In  support  of  this  statement, 
we  have  participant ' opinions  on  the  weekly  opinionnaire  to  the  effect  that  the 
•staff  was  available  to  talk  Co  whenever  necessary,  particularly  from  the  third 
week  on.     (During  weeks  one  and  two  of.  the  Institute  four  participants  were  less 
than  satisfieW^ith  staff  availability  but  from  that  time  on  everyone  agreed 
that  the  staff  was_  indeed  readily  available .)     Similarly ,  partic ipants  said  the  ' 
staff  was  paying  enough  attention  to  the  important  probUms  that  arose  although 
a  couple  of  participants  were  never  satisfied  in  this  respect.     Participants  also 
felt  that  the  staff  had  taken  appropriate  actions  to  resolve  problems  as  they 

as "Good"  or  ''Excellent''  while  three  rated  them  as  ''Poor''  b^  Briefly, 
then.:,  it  seems  that  the  Instdtute  staff  ;  did  an  excelle 

available  :  to  lis  ten  to-and  at  leas  t  gave  the  appea^nce  of  ac  t  ing  to  correc  t-  ; 

An  important  component  of  the  duties  of^  the  staff  of  a  summer  instirut^.. 

however,   is  to  facilitate  the  smooth  flow  of  the  instructional  con^ 
institute.:  this:  IS.PARTICULAl^^ 

IN;  NEW  HAMPSHIRE  ;  WHERE  I'Ara'iCIPANTS -IN  ^     TOTAL  OF  SIX  ;WEEKS-  AEE,  EXPOSED^^  TO  / 
SEVERAL  HIGH-POt^RED  •'EXPERTS,^  TOPIC  TO:  PRESENT  M^  '  ^^  ^ 

A  UNIQUE  METHOD  FOR  PRESEOTING  THAT  CONTEOT.  .  If  the  presentations  of  guest  '  ^ 

consultants  are  to  be  "relevant-'  to  the  the^^^  of  the  institute  and  to  the  needs  V  . 
of  the ;  i^articipants ,   if  the  transit  ion  from  consultant  to  consultant  (topic  'to 
topic)   is  to  be  accomplished  smoothly ,  and  if  time-consuming  and  ■threatening.        : / 
confrontations  of  participants  _and  consultants  are  to  be  miaimi^ed.   it  is  imperative 
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that  the  permanent  administrative  sta.-^f  of  the  institute  take  strong  positive 
action.     IVhat  kinds  of  actions  might  serve  to  facilitate  the  flow  and  transition 
of  instniction?     First,   it  would  seem  necessary  that  participants  be  given  so^ne 

each  consultant.     This  might  include  an  overview  of  the 
content  of  his  preseritation  (perhaps  consultants-as  a  part  of  their  responsi- 
bilities to  the  institute-should  prepare  such  overviews  themselves)  several 
•     days  before  the  consultant  is  scheduled  to  "perform. "    1,.:  addition,   it  might  • 

help  the  participants  if  they  were  to  receive  a  brief  vita  of  each  consultant . 
:  Activities  like  these  last  two  would  give  the  participants  some  idea  about  what  . 
the  consultants  were  going  to  present,   thereby  alleviating  some  of  the  anxieties. 
(That  such  anxieties  among  participants  did  in  fact  4ist  i^^^ 

Institute  is  reflected  in  reports  from  the;  participant  observers  during  the  early  • 

parts  of  the  Institute  and  in  some  of  the  comments  made  to  the  evaluators  during  \ 
; :  site  visits .)  ■  .',  .  _ 

id-  Equally  as  important  as  preparing  the  institute  participants  for  the  : 

consultants  is  the  task  of  ;pre^ 

is  ,  consultants  should ;  be  provlded-in  advance-with  infom.tion  aboMt  the 
institute  participants:     •:beir  backgrounds,  job  descriptions,  purposes  in  att 

^   the  institute,  and  any  inforn^ation  available  about  what  to  expect  by  way  of  group 
dynamics  during  the  ins titute.     (This  latter  item  will  be  most  useful  to  the  .second 

:■;  and  subsequent  consultants.)  - 

If  the  kinds  of  information  described  in  the  previous  tw^^ 
•    had  been  available  during  the  New  Hampshire  Institute,   it  is  likely  that  many 
of  the  consultant-participant  conflicts  described  below  could  have  been  avoided. 
It-  is  on  this,  issue  that  the  administrative  staff  :of  the  New  Hampshire. ;  Institute  ; 
■should  be  taken  to  task.     Truo.   the  Institute  staff  was  h^ 
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of  .time  between  date  of  funding  and  the  start  of  the  Institute.    True,  the 
Institute  director  s tepped  in  at  the  last  minute  to  replace  the  person  who  had 
planned  the  Institute,  with  a  very  different  philosophy  and  a  very  different 
administrative  style.:    Nevertheless,  the  important  task  of  preparing  participants 
for  the, consultants  and  consultants   for  the  participants  could  have  been  better 
handled.     The  lateness  of  funding  undoubtedly  was  a  factor  here. 

In  spite,  of  the  above  conttnents,  this  report  would  not  be  complete  or 
fair  without  mention  of  one  outstanding  counter-example.     After  two  and  one-half 

weeks  of  the  Institute,   it  was  apparent  that  consultants  could  riot  come  in  and 
"do  .their  thing"  for  their  alloted  time  without  careful  consideration  of  the  needs 

:and  desires  of  the  participants^/  The  first  two  consultants ,  Cook  and  Walbesser, 
.  had  arrived  with  j  ust  that  intent  ion ,  with  almos t -disas  trous  results On  the 

basis  of  the  experiences  of  Gook  and  Walbesser,   the  Institute  director  made  a 
:p^rso:nal  visit  to  the  next  consultant,  Daniel  Stuff lebeam,  several  days  in  advance^^ 

of  his  scheduled  participation  in  order  to  describe  to  him  the  kinds  of  ditfi-  . 

warning;  Stuff  lebeam  made  some  drastic  changes  in  his  method  of  :  operation|chariges 
which  possibly  "saved"  the  Institute.  :  ' 

:         The  staff  was  also  remiss  with  respect  to  severaL  other  adminisitrative 
details .     For  example it  was  a  common  experience  of  both "  consultants  and  eval-.  - 
uators  to  arrive  on  Sunday  night  with  little  or  no  information  about  what  to 
expect  Monday  morning/    As  an^£^^ 

England  Center  for  Continuing  Education  at  1^  Sunday  evening  in  the  belief 

t^at  he  was  tc.  reside  there  for  the  next  two  weeks  only  to  find  that  an  fact  -he 
did:  not  ,  have:  a  . reservation  there.  A 
that  time  elicited  the  information  that  he  would 
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be  Staying  elsewhoro'.     At  the. 


-time  of  his  arrival,  Stufflebeam  also  did  not  know  where  he  was  to  be  on  Monday 
or  at  what  time.     It  should  not  be  necessary  for  a  consultant  to  call  the  Institi. 
director  at  that  late  moment  to  find  out  such  basic  information. 

Institute  participants  were  themselves  somewhat  aware  of  this  weakness 
in  the  administration  of  the  Institute,     Wien  asked  if  they  thought  the  Institute 
was  well-organized,  participants   responded  generally  favorably  but  noticeably 
less  so  than  to  other  qviestions  about  the  staff  discussed  above. 

STAFF:     PARTICIPANT  OBSERVERS 
,   Two  participant  observers ,  one  black  and  one  white,  were  recruited  for 
the  Mew  Hampshire  Inst itute .     Their  primary  allegiance  was   to  the  Institute 
director  although  it  was  anticipated  that  their  observat ions  would  also  be  of 
considerable  use  to  the  evaluatprs •     That  they  fulfilled  their  primary  role-- 
that  of  monitoring  and  reporting  to  the  Institute  director  the  feelings  of  the 
participants  about  the  Institute--can  be  attested  to  by  the  administrative  staff. 
That  they  were  of  less  use  than  anticipated ' in  their  secondary  role--that  of 
P^^^^^?-d2.nS  usef ul  information  to  the  evaluaticn  team  during  the  Iris t itut^^^^ 
be  attested  to  by  the  cvalvntors       This   lacl:  of  Mtil?t:v  to  evaluacors  o:   z'.- ^ 

participant  observers   raises  some  important  questions  about  how  evaluators  can 
obtain  the  most  benefit- -or  any  bene  fit  — from  the  presence  of  such  observers. 
These  questions  will  be  considered  later  in  this  report • 

An  important  factor  to  consider  in  the  use        participant  observers  ■ 
is  the  careful  specification  of  their  role  in  an  institute.     Are  they  to  be  pri- 
marily, participants?    Or  primarily  observers?     It  is  likely  that  this  question 
will  answer  itself,  given  the  context  in  which  .the  partic ipant  observer  is  func- 
tioning  and  his  personality.     It  is  also  important  to  consider  the  unwanted  side 
effects;  of  the  presence  of  participant  observers.     In  the  New  Hampshire  Institute 
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there  was  some  early  resentment  on  the  part  of  some  participants  to  the  prepx^jnce 
of  the  observers,  especially  in  the  residential  setting.     As  one  participant 
remarked,   "It  gets  annoying  to  have  someone  looking  over  your  shoulder  while 
you're^^ting  dinner,  noting  who  you  are  talking  to  and  what  you  are  saying." 
Yet,  from  the  point  of  view  of  the  Institute  staff ,  the  information  gleaned  by 
a  participant  observer  in  that  setting  was  and  is  an  invaluable  asset  to  on-going 
operations  and  administrative  decision  making.     Related  to  this  last  point,  on: 
the  Summary  Op inionnaire  f if teen  partic ipants  rated  the  participant  observers  as 
good,  eight  were  uncertain,  and  the  other  five  gave  decidedly  negative  responses . 
In  the  context  of  generally  positive  reactions  to  the  Ins t itute ,  these  responses 
represent  what  might  be  significant  dissatisfaction.     Unfortunately,  perhaps,  it  : 
is  impossible  to  differentiate  between  the  two  observers  to  determine  whether 
either  or  both  represented -a  source  of  dissatisfaction. 

.■        CONSULTAOT^  .  ..  . 

=     :  'The  prospectus  for  the  New  Hampshire  Ins  tit  ute  indicated  five  major  - 
instructional  components:     Week  6ne--Desmorid'  Cook  on  proiect  management  models 


and  techniques  ;  ■•^3ek 


cs  T-JO  ■  and  '  Three'-'-H 


objectives,  design  of  assessment  tasks ,  and  construction  of  learning  hierarchies; 
Weeks  Four  and  Five—Daniel  Stufflebeam,  Egbn  Cuba,  and  Robert  Hammond  on  evalua- 
tion models  and  techniques ,  seminars  on  problematic  simulations;  and  Week  Six-- 
William  Asher  and  Daniel  Heisey  on  problems  associated  with  developing,  disseminat- 
and  adopting  educational  innovations This  lineup  was  described  in  the  planning 
group  meeting  prior  to  the  Institute   (May  24)  as  a  "parade  of  stars This 
particular  model   for  a  sutraner  Institute  created  some  unique  problems.     The  present 
section  of  this  report  is  intended  to  document  some  of  those  problems  and  offer 
some  suggestions  for  avoiding  them  in  other  Institutes. 
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(In  preparing  this  section  of  the  report,   it  was  virtually  i„,possible 
to  separate  the  various  components  o£  the  Institute:     consultants,   the  content 
of  their  presentations,   the  sequence  in  which  they  appeared,  the  nature  of  the 
participant  group.   U.e  actions-or  inactions-o£  the  administrative  staff.  As 
a  consequence,  although  the  section  is  headed  ••Consultants       i,s  scope  is  much 


broader,) 


:  !  It  Is  th.  impression  of  tl,e  evaluators  tha.  many  of  the  probW  arising 
fro„  eo„suUan.-participa„t  or  .onsultant-staf f  encounters  in  the  Ne„  Hampshire 
Institute  resulted  fro.  the  fact  that  the  two  group  .oubinat ions  held  very  dif- 
ferent sets  of  expectations  as  to  what  tho  Institute  was  all  about .     The  consu 1 tants. 

who  had  had  much  experience  with   rtrpHf  m-^t^^i  ^  -  . 

  ^  ''^'"'"■'""=l>  institutes,  came  with  the 

.expectation  that  they  were  going  to  spend  one  or  two  weeks  ..doing  the ^  Z 
That  is  .  both  Cook  („eelc  One)  and  :«aibesser  Cheeks  Two  and^  Three)  anticipated:  '  ■ 

coming  in,  presenting  their  materials  in  their  own  ways,  and  leaving.     Thepar--  : 
ticipants,  many  of  whom  had  not  been  initiated  into  the J^rate rni ty  of  su^er- 
institute  attenders.  came  looking  f or  mate  rial  which  woUld  be  directly  relevant  '  r 
to:  them  S  they  perfo^ed  in  their  professional  roles  ..back  ho.e...    Because  the  ■  • 

participants  ,^  ens  a  ,  groun     ■•t-r.-.  \,  .  .  .  ^ 

V  -  •  : '■.^^^--o-^eous  witii   i-especL   to  what  was  ' 

"relevant  to  them...  their  individual  expectations  were  impossible  of  fulf indent 
in  the  context  of  the  Inst itute .     Because  the  first  two  consultants  were  not  ^ 
initially  aware  of  (1)  the  diverse  nature  of  the  group  and  (2,  the  expectations 
Of  the  participants,   they  had  no  chance  even  to  attempt  to  meet  those  expectations.} 

on  the  morning  of  the  third  day  of  the  first  week  of  the  Institute 
Oes^nd  cook  was  directly  challenged  by  one  of  the  participants.     Oe  was  asked 
to  justify  his  presentation  of  the  material  he  was  dealing  with ,  to  indicate 
itsimportanceand: relevance  to  the  participants.     This  open  challenge  ca^e.o     '  ^ 
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be  a  pattern  for  the  remainder  of  the  Institute.     In  ret  rospect ,   it  was  readily 
predictable  that  each  consultant  would  find  it  necessary,  early  in  his  presentation, 
to  meet  and  deal  with  a  direct  challenge,     Tlr\s  confrontation  occurred  with  each 
of  the  consultants,  without  exception,  although"  it  took  a  more  violent  form  for 
those  who  had  made  no  apparent  attempt  to  list^^n  to  or  consider  plausible  adaptive 
strategies  based  on  the  characteristics  of  the  participants  than  for  those  who 
were  obviously  making  a  real  effort  to  acccr^vndate  the  needs  and  desires  of  the 
participants . 

Of  particular  interest  is  the  various  strategies  the  consultants  used 

in  their  attempts  to  deal  with  these  confrontations .     Cook  had  been  warned  in 

advance: to  expectva  confrontation  (a  "plus"  for  the  monitoring  function  in  this 

Institute).     On  the  basis  of  his  perceptions  of  the  participants  \  feelings ,  he  \,^- 

altered  his  instructional  strategy.     In  his  own  words: 

'JInstructional  approach  was  modified  so  that  more  informality 
and  participation  were  generated.   \  I  put  less  emphasis  oh  / 
covering  material  and  dealt  with  the  idea  of  trying  to  ""accom- 
:plish  'only  two  or  three  major  points  ....     I  think  this  helped 
to  ease  tens  ions  during  the  last  couple  of  days,"   

(V/hy  did  the  confrontation  occur  in  the  first  place?     As  Cook  saw  it,   there  were 

two  factors  involved: 

'Tirst,  a  tendency  on  my  part  to  go  with  past  success  in  view 
of  not  knowing  the  full  composition  of  the  group.  Second, 
the  heterogeniety  of  the  group  in  both  background  experiences 
and  educational^  level. "     (Underscoring  added) 

Again,   it  seems  that  the  Institute  staff  could  have  taken  steps  to  mihimize  these 
problems .)  : 

•       Desmond  Cook  had  ^Mone  his  thing"  with  program  management .  '  In  came 
Henry  Walbesser  with  a  different  instructional  style  to  present  behavioral  objec- 
tives and  learning  hierarchies.     One  would  hope  that  the  Institute  staff  could 
have  profited  from  Cook's  experiences  and  would  have  effectively  sensitized 
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Walbesscr  for  the  group  he  was  about  to  encounter.     Unfortunately,   this  did  not 
occur.            it  did  not  occur  we  cannot  say.     It  may  be  that  Walbcsser  was 
unresponsive  to  input   from  the  Institute  staff;   it  may  be  that  the  Institute- 
staff  failed  to  act  strongly  enough  ;   it  may  be  .  . .  .  :   

Institute  participants  resented  Walbesser's  coming  in  and  presencing 
a  poorly  edited  prograniraed  instruction  text,  a  preplanned  program  of  instruction 
which  included  major  componenLS; of  large-group  lecture  and  what  the  par t ic ipancs^ 
saw  as   "busywork"  exercises.      (Several  participants  noted, "Had  I   (they)  wanted 
to  study  Walbesser's  text,  I  (they)  would  have  mailed  S2. 25  plus  postage  and 
read  it  in  the  leisure  of  home/of f ice. "    The  feeling  was  that  H.   w.   and  his  team 
did  not  supplement  the  text  with   their  presence !)  ,    Once  again,   the  differing  evpec- 
tations  of  consultant  and  participants  seemed  to  .underlie  the  conflict .  Walbesser 
was  less  adaptive  than  Cook.      (If  the  leadership  personnel  in  the  .Institute  had 
done  an  effective  job  of  coordination  and  preparation,  he  might  not  have  found 
it  necessary  to  modify  his.  approach  once  he  was  under  way.)'    Participants  reported 
that  Walbesser  tried  at  lease   four  different  "cop-outs'-  in  respondihg  to  ch.alleaae : 

1.     Threatening  part  ic  ioants    (son-.oth  in^  ^^'Zf^-.-^le':'^ 

1  ajj-,j;'.ea;  or       i  i.  vri.ce  your  boss  ,  your  ■ 
Congressman,  H.E.W."); 

.2.     Asking  the  Institute  staff  to  send  two  participants  home 
and  cut  off  their  s  tipends ; 

3.  Declaring  that   there  was  a  "racial  problem"  among  Institute 

participants   (NOTE:     staff  and  participant  observers  could  ' 
find  :;no  evidence  of  such,  a  problem)  ; 

4.  "Apparently"  changing  his  style  for  his  second  week  (by 
having  lectures  only  on  Monday  and  Friday  mornin,?TS  and' 

■-        "free-wheoling  seminars"  the:  rest  of  the   tiivie.-  fnG" 

seminars  v.-ere   "the  sane  old'llenry  with  .^mailer  groups ")  . 

Given  that  it   is   imposr;ifale  to  control   the  personality  style  o  t  ■  consul  tants,   i  t: 
is  imperative  that  the  administrative  staff  of  an  Institute-  such  .-,s   this  one  work 
''tRiC'''''  ""^^"^^^^^^  :  °I  consultants  are  prepared  for  theru. 
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Enough  of  anecdotal  data  for  now.     How  did  the  part icipants  themselves 
react  to  these  events?    Data  relevant  to  this  question  can  be  gleaned  from  the 
Weekly  Participant  Opinionnaires .     First,  did  participants  enjoy  the  instructional 
staff  (consultants)  each  week? 

Yes      Yes  but . • • .  No 

Week  One  (Cook)  16  9  2 

Week  Two  (unmcMlified  Walbesser)     15  2  7 

Week  Three  (modified -Walbesser)     19  3  4 

Week  Four  (Stufflebeam)  25  2  - 

Week  Five  (Stufflebeam,  Cuba,        27         '    -  - 
and  Hainraond 

Week  Six  (Asher,  Heisey)  NO  DATA  COLLECTED 

Clearly  from  this  point  of  view, : Week  One  was" not  had.  Week  Two  not  too  good. 
Week  Three  pretty  good,  'and  Weeks  Four  and  Five  excellent. 

But  there  are  other  aspects  of  these  instructional  blocks  which  must 
be  considered.     I^Jhether  or  not  a  participant  enjoyed  the  consultant  (s)  for  a 
particular  week,  did  he  learn  a  lot?  Observe: 

Yes:  ■    l:es  .  bu c ,  , ,  Wo, 
Week  One  17  4  3 

Week  Two  13  3  6 

Week  Three  11  5  8 

Week  Four     .  20  5  - 

;   Week  Five  27  •  -  V 

As  anticipated,   there  is  a  good  bit  of  cons  is tency  in  the  distribution  of  responses 
to  these  two  questions.  .'  But  notice  the  data  for  Weeks  Two  and  Three.  Participants 
"enjoyed"  Walbesser  more  during  Week  Three,  but  more  of  them  felt  they  "learned  a' 
lot^'  during  Week  Two.     This  might  be  cons ide red  supporting  evidence  for  the  earlier 
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assertion  that  V/albosser's  change  in  procedure  was  more  a  *'cop-out"  than  a  real 
attempt  to  accommodate  himself  to  the  group. 

/  It  was  during  the  third  week  of  the  Institute  that  the  director,  Elwell, 
met  with  Stufflebeam.     As  a  result  of  that  meeting,  Stuff lebeam  made  an  effort 
to  adapt  to  the  needs  and  desires  of  the  part icipants .     Among  his  more  success ful 
tactics  were  (1)  requesting  that  the  partic ipants  tell  him  what  kinds  of  problems 
they  were  concerned  with  and  would  like' help  on;   (2)  setting  up  small-group  sessions 
focusing  on  common  problems;  and  (3)  appointing,  after  consulting  with  the  Institute 
staff  on  the  Sunday  evening  preceding  his  first  instructional  day,  an  "advisory 
group"  of  participants"  to  assist  in  planning  the  week's  activities.     One  of  the 
evaluators   (Wardrop)  was  present  during  Stufflebeam's  initial  encounters  with  the 
participants.  .  It  was  that  evaluator's  opinion  that ,  once  Stufflebeam  had  taken 
those  three  initial  actions ,  he  could  proceed  in  whatever  manner  he  xhose  with 
the  full  compliance  of  the  participants^    (That  the  tactics  were  successful  is 
attested  to  by  the  participants  I  responses  on  the  fourth  Weekly  Opinionnaire .) 
Few  r^a!  consultant-participant  difficulties  arose  during  the  remainder  of  the 
Institute  eiluhough  it  should  be  noted  that:  seve  ral  . partic  ipaai::^   '^cop J-out/' 
during  the -sixth  week  by  avoiding  contact  with  the  instructional  team*  One 
participant,   in  fact,  left  the  Institute  on  Wednesday  of  that  last  week, 

:\  y-:'^^'-:TABiic^^ 

\  A  leitmotiv  throughout  the  New  Hampsh ire  Inst  1 tut e  was  the  interplay  ; 
of  the  diversity  of  backgrounds ,:abil it ies,   responsibilities ,  and  concerns  of  ' 
the  participants.     Such  heterogenie ty  was  built  into  the  Institute  by  the  criteria  : 
for  selection  of  participants  in  the  original  proposal.     The  Institute  was:  to  / 
serve  graduate  students,;  LEAs ,  and  personnel  in  state  and  federal  agencies . and  was  / 
to  include  specifically  ten  southern  blacks.     As . a  consequence  of  the  selection  v 
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process,  the  Institute  wound  up  with  2^  talented  (for  the  most  part) ,  concerned  > 

(without  except  ion) ,  and  aggressive  (in  comparison  with  other  groups  in  previous  ■ 

institutes)  participants .     Such  a  mix  was  perhaps  the  single  most  Important  causal'Y 

factor  in  the  unfolding  story  of  this  Institute.     It  created  problems  for  the  ;  ' 

consultants  from  the  very  beginning: 

[One  of  the  Institute's  greates t  weaknesses  was j  the  heter- 
ogeniety  of  the  group  in  both  background  experiences  and 
educat ional  leve Is .   (Consultant  A) 

[One  of  the  Inst itute 's  greatest  weaknesses  was ]   the  diversity 
of  the  purposes  for  which  the  participants  were  attending  the 
Institute....   (Consultant  B) 

[One  of  the  Institute 's  greatest  weaknesses  was]  hetero- 
geneous participants.     (Consultant  C) 

[One  of  the  Institute 's;  greatest  weaknesses  was  J   too  much  . 
variance  among  the  participants  in  both  motive  for  attending 
*  and.  acadernic  ability.     (Consultant  D)  ^ 

Based  on  experiences  in  this  Inst  itutej  (and  elsewhere)  ,  we  feel  impelled;^ 

to  raise  some  important  ques tions  about  the  nature  of  the  select  ion  process .      i   I'  ; 

These  questions  will ' be  cons ide red  in  a  later  sec tion  of  this  report • 

;  INSTRUCTIONAL  CONTEOT  ^ 

Just  as  diversity  characterized  the  participants  in  the  New  Hampshire* 
Institute,  so  the  cry  for  relevance  characteri^ed  their  feelings  about  the  instruc- 
tional program  of  the  Institute.     'Program  management, ""behavioral  objectives," 
"learning  hierarchies,"  "the  CIPP/ evaluation  model,"  and  "comprehensive  achieve-  .: 
ment  monitoring"  are  fashionable  among  educational  researchers  and  theorists ; 
but  some  participants  in  the  New  Hampshire  Institute  could  not  see  the  possibility  : 
of  applying  these  techniques  and  procedures,  to  the  problems  with  which  they  wore 
.  themselves  concerned.     They  did  not--many  of  them,,  at  least--want  their  time      .  . 
"wasted."    Wiether  this  was  the  feeling  of  the  majority  of  the  participants  does 
not  matter  for  it  was  the  feeling  of  at  least  a  very  vocal  group.  . 


if 

-15-       .  ,    .  ,  ■ 

This  cry  for  "relevance"  created  some  unique  problems  because  what 
was  relevant  for  any  half-dozen  participants  was  not  for  the  remaining  twenty- 
two.     Given  the  diversity  of  the  participants,  satisfying  a  demand  for  relevance 
of  the  instructional  program  simply  was  not  possible.     The  only  course  of  action 
which  had  hope  of  succeeding  was  individualization  of  instruction.     But  total 
individualization  was  itself  not  feasible,  given  the  limited  amount  of  time  and 
:    energy  of  any  one  of  the  consultants  .     Tt  was  most  nearly  approached  during  tlie 
fifth  week  with  SCufflebeam,  Cuba,  and  Hammond  as  an  effective,  instructional 
team.     Inadequate  kndwledge--on  the  part  of  both  staff  and  consultants --of  the 
backgrounds  and  interests  of  the  participants  was  another  limiting  factor*  The 
participants  and  participant  observers  were  themselves  aware  of  this  limitation, 
as  evidenced  by  their  suggestions  for  ''pretesting"  the  participants  in  order 
to  place  them  in  small  groups  for  certain  parts  of  the  instructional  program. 

Both  the  cry  for  relevance  and  the  desire  for  individualized  or  small- 
group  instruction  had  their  counterpoints,  however.     In  the  former  case  it  is  impor 
tant  to  keep  in  mind  the  viewpoint  expressed  by  one  of  the  participants  (address- 
ing Stuff  lebeam):     "I  came  here  to  learn.., to  hear  and  learn.     I  might  not  ; 
know  what  you ' re  talking  about,  I  might  not  understand  you,  but  whenever  you/ re 
talking,  1*11  be  sitting  there  lis tening . "    For  people  like  this  one,  the 
exposure  to  the  "stars"  was  important .     Relevance,  for  him,  was  not  a  primary 
criterion.     In  the  latter  case  (individualized  or  small-group  instruction) ,  an 
=    interesting  and  puzzling  phenomenon  occurred:     When  Stuff lebeam  in  Week  Four 
;     strove  valiantly^,  to.  divide  the  participants  into  small  groups ,  with  at  least  a 
■v   modicum  of :  commonality  of  interest  and  concerns  within  each  group,  he:  meti;  wi th 
:  v;  res  j;He  found  that  only  If  he  personally  would '  meet  with<  every  small^^^^  v^^^^^ 

O   )up  would  many  ;  of  the  :  part  icipants  raccis?       that  approach  to  ins  true  tion;  ■ 
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Everyone--or  nearly  everyone-- insisted  on  being  where  Stuff Icbeam  was  Lest  they, 
"misa  something. Does  this  phenomenon  have  some  impl  ications  for  the  kinds  of 
selection  strategies  and  criteria  which  ought  to  be  built  into  institutes? 

Questionnaire  responses  provide  further  information  about  the  reactions 
of  participants  to  the  content  of  the  instructional  program.     Perhaps  the  most 
important  question  in  this  realm--on  a  week-to-week  basis,  at  least--asked  if  the 
wc  .    3  content  had  been  challenging  and  important.     The  week-by-week  responses 
were  as  follows:  . 

Yes      Yes  but...     No    Don't  know 

Week  One   (Cook)  18  9  -  - 

Week  Two  (Walbesser)  15  6     -        3  1 

;  Week  Three  (Walbesser)  14  7  6  2 

Week  Four  (Stuff lebeam)  21  5  -  - 

Week  Five  (Stuff lebeam,  Cuba,     24  -  -  1 

and  Hammond) 

With  the  exception  of  Wesks  Two**and  Three,  all  participants  responding  lo  the  quest 
naires  felt  the  material  presented  was  both  challenging  and  important J^;  Even  during 
those  two  exceptional  weeks,   three-fourths  of  the  participants  agreed  with  this 
statement,  •^irtually  the  same  distribution  of  responses  was  found  for  an  indirect 
question  about  the  relevance  of  the  instructional  content  each  week;  viz .  "Would, 
yourprofessional  peers  back  home  be  interested  in  what  you  learned  this  week?" 

■Several  questions  in  the  weekly  opinionnaire  were  designed  to  elicit 
participant  self-evaluations  of  their  learning  each  week.     After  all,   foir  partic- 
ipants to  benefit  from  the  instructional  content  of  an  ins titute,   they  must  master 
at  least  .part  of  that  content.     The  first  of  these  quest ions--"Di d  you  learn  a 
lot  this  week? "--yielded  the  following  responses: 
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Yes 

Yes  but . . , 

No 

Don't 

Week 

One 

17 

'  4 

3 

3 

Week 

Two 

13 

3 

6 

2 

Week 

Three 

11 

5: 

8 

2 

Week 

Four 

20 

5 

2 

Week 

Five 

27 

(The  pattern  is  familiar  by  now,)  As  a  participant,  you  may  feel  you  have  learned 

a  lot  but  "Do  you  think  you  could  teach  another  group  what  you  have  learned  this 
week? " 

Yes      Yes  but.,.     No  Don't  know 

Week  One  11           10              2  2 

Week  Two  14              3              \  3 

Week  Three  12              7              7  1 

Week  Four  18             3              3  3 

.  Week  Five  9            12              2  2 

■  '       "  ■        ■  \       '  '  ■  ■  i         "         .  '  .  .  ■ 

The  participants  are  telling  us  something  here.     Yes,  they  have i indicated ,  we  have 
learned  a  lot.     No,   they  continued,  "we  are  not  as  confident  that  we  could  teach 
it  to  someone  else.     (Week  Five  deserves  special  attention  in  this  regard.  The 
primary  instructional  activity  -that  week  was  an  elaborate  simulation  exercise,  one 
which  would^  require  much  greater  understanding  of  content  and  a  not-insignificant  " 
set  of  materials  in  order  for  anyone  to  attempt  to  use  it  with  [another  group .  It 
IS  not  at  all  surprising  that  the  participants,  who  were  unanimous  in  indicating 
that  ,  they  had  learned  a  lot,  were  much  less  confident  that  th(?'y  personally  could 
teach  what  they  had  learned  to  others.) 

■  ..Otl^er  ques  tions  on  the  weekly  opinionnaires  we  re  de;S  igned  t  o  get  at 
participants'  affective. reactions  to  the  week's  instruction,  /  Paralleling  the 
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earlier  question  about  participants'  ability  to  teach  the  week's  content  to  another 
group  is  the  question  which  asks  "V7ould  you  want  to  teach  another  group  what  you 
have  learned  this  week?"    By  week,  the  responses  to  this  item  were: 

Yes      Yes  but ...     No    Don't  know 
Week  One  16  5  -  5 

Week  Two  12  2  6  4 

Week  Three  12  2  8  4 

Week  Four  13  9  2  2 

Week  Five  18  4  2  2 

There  is  a  great  deal  of  agreement  between  responses.        this  item  and  responses 
to  the  earlier  one.     There  are  a  couple  of  notable ; except  ions  .  however .  Partic-- 
ipants  were  generally  uncertain  about  their  ability  to  teach  the  materials 
presented  in  Weeks  One   (Cook)  and  Five   (Stuff iebeam.  Cuba,  and  Hammand 's  simula- 
tion); but  in  both  cases  they  expressed  a  desire  to  do  so.     In  contrast,  partic- 
ipants were  ,  as  a  group  .  more  confident  that  they  could  teach  the  material  of 
Week  Four  (the  Phi  Delta  Kappa  approach  to  evaluation)  than  they  were  interested 
in  doing  so.  . 

^Another  approach  to  obtaining  affective  reactions  to  the  material 
presented:was  through  a  question  abo%.r:  participants'  willingness  to  recornmend  ^  ^ 
that  the  materials  be  used  in  other  summer  institutes: 

Yes      Yes  but...     No    Don't  know 


ERIC 


Week  One 

19 

4 

2 

Week  Two 

6 

7 

4 

Week  Three 

13 

6 

Week  Four 

5' 

3 

Week  Five 

,  1  ■  "  - 
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It  is  apparent  that  the  materials  from  Weeki^  Two  and  Three  were  unpopular.  How 
much  of  this  reaction  should  be  attributed  to  the  participants'  negative  reactions 
to  the  consultant  for  those  weeks  is  of  course  impossible  to  detennine.     It  should 
be  noted,  however,   that  some  of  the  comments  we  received  were  directly  critical 
of  the  materials  themselves.     ("Turned  off  by  instructional  material. "Redundant 
and  unimportant."    "Needed  revision  and  re -editing • ") 

In  an  institute  charactt^rized  by  a  "parade  of  stars,"  special  attention  . 
must  be  paid  to  the  problem  of  t  rans  it  ion  f  rom  one  topic  to  the  ne>:t,   from  one 
consultant  to  the  next.     That  this  need  went  unmet  in  the  New  Hampshire  Institute 
is  clear.     When  asked  if  the  transition  from  previous  weeks  of  the  Institute  was 
satisfactory,  participants:  at  the  end  of  Week  Two  (the  transition  from  Cook  to 
Walbesser  at  the  beginning  of  that  week)  were  evenly  split :     10  said  it  ;  was ,  10 
said  It  was  not ,     Similarly  /  the  transition^:  from  Weeks  Three  to  Four  (from 
Walbesser  to.  Stuff lebeam)  was  seen  as  "satisfactory"  by  15  participants  and  as 
"unsatisfactory"  by  10  others .     Throughout  the  Institute,  partic ipants  were 
uncertain  about  what  to  .expect  from  each  new  consultant ,  consultants  were  unaware 
of  vhat  their  predecessors  had  done  or  what  their  successors  were  goine;  to  do. 
Once  again,   the  administrative  staff  of  the  Nev;  Hampshire  Institute  must  be  t.aken 
to  task  for  failing  to  handle  this  very  important  aspect  of  the  Institute. 
[Note:     Consultants  did  not   (in  general)   share  materials/outline  as  "promised" 
at  May"24,  Wsshington,  D.  C\  conference.] 

Pai   icipants. were  asked  at  the  end  of  the  Institute  to  make  some  judgments 
about  the  relevance ,.  value ,  and  transportability  of  the  various  components  of;  the 
Institute.     For. .each'  of  the  four  components  of  the  Ins titutey  ovqir  three-fourths; 
of. the  participants  agreed-  that  the  material  presented  coula  uavc  considerable 
applicability .  (this  in  ;s  all  the  concern  about  relevance)  and:  that  they!  : 
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personally  would  be  able  to  put  the  materials  to  good  use.     The  majority  of  the 
participants  also  felt  that  the  materials   for  ail  components      'copt  that  involving 
StL .  (Weeks  Four  and  Five)  would  be  "readily  transferrable  to  other  Institu 

as  an  instructional  package  without  requiring  that  the  consultant  (s)  who  presented 
it  here  be  involved."     In  direct  contrast  to  this  reaction,   fewer  than  half  of 
<:he  palrticipants  "would  recommend  that  my  colleagues  attend  a  Workshop  or  Institut 
which  had  this  topic  as  its  central  focus"  except  for  that  component  involving 
V7eeks  Four  and  Five.     For  this  presentation  on  evaluation  models  and  techniques, 
all  but  ir^ne  of  the  participants  responding  to  the  Summary  Opinionnaire  would 
encourage  colleagues  to  attend  such  a  workshop  or  institute. 

THE  I>^^v;TITUTE  as  A^vJlOLE 
■  \   This  section  is  simply  a  cataloging  rif  i*nme  of  the  summary  impressions 
of  the  participants  about  the  New  Hampshire  Institute.     First,  the  Institute 
achieved  its  objectives  "fairly  well"  or  Wery  well"  for  near ly  all  part ic ipants . 
Secondly,   two  weeks  of  behavioral  objectives/learning  hierarchies  was  too  long, 

as  was  one  week  of  Asher  and  Heisey,     (More  about  that  week  later •)    'In  retro- 

f  . 

sp'-^ct,  TTiaterials   in  all  com-nonents  vf?re  seen  as  somewhat -to-very  rolovant  '^.-lt:l'':^r^ 
the  behavioral  objectives/learning  hierarchies  component  was  rated  notably  lower 
than  the  other  three  in  this  respect). 

Participants-  also  felt  that  the  content  presented     ^is  both  practical 
and  feasible  and  that  their  competencies  to  conduct  evaluation  studies  and  to 
develop  and  :^,mp I ement  sound  decision-making  strategies  had  been  enhanced.  Finally 
aft  <^r  -all --well ,  almost  all--was  said  and  done,  only  two  part  ic  ipants  indicated 
that,  r>v  '.r,  overall  reaction  to  the  Institute  was  unfavorable.) 
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THE  SIXTH  WEEK 

Why  don't  we  stop  fooling  ourselves? 
The  game  is  over. 
Over, 
Over . 

No  good  times,   no  bad  times ^ 
There *s  no  times  at  all.... 

--Paul  Simon,  "Overs" 

"Major  emphasis  of  this   [sixthj  week  will  be  on  the  study  of  provic'ins 

e  actual  leadership  within  an  educational  institution  in  attempting  to  promote 

re ful  study  change  as  a  result  of  carefully  constructed  research  and  evaluation 

forts.,.."     (Project  proposal).     In  practice,  the  sixth  week  consisted  of  two 

rnponents:      (1)  soiT^what  formalized  presentations  of  poorly  prepared  (if  at  all) 

teria!  by  the  consultants   (Asher  and  Heisey) ,  which  we  re  barely  tolerated  by 

st  participants;    (2)   individual  conferences  with  many  participants  by  the 

nsultants,  at  which  time  the  consultants  provided  help,  advice,  and  criticism 

"research"  proposals  the  participants  were  developing .     This  latter  activity 
5  reasonably  well  received  by  many  participants . 

Some  indications  of  the  general  indif ference--if  not  actual  negativism- - 

the  part:3.c  ip  ;:int:3   t:ov;:ird  this   final  w^ek  ^  5  a'c  L  ivic  ies.  v/e  re  ootaiaec  from  the 
nmary  Opinioanaire.     Twelve  part icipants   (43  per  cent)   indicated  that  one  week 

Asher  and  Heisey  *s  efforts  wa5   too  long .     Only  14  participants   (50  per  cent) 
Lt  thxs  final  week's  instruction  ITa^T^^  be  applicable  to  their  own  work., 

^^lly,  only  12  participants   (43  per  cent)  would  recommend  to  their  colleagues 
It  they  attend  a  workshop  or  institute  which  had  this  week's  content  as  its 
-mary  focus ,     If  an  intended  outcome   for  this  week  was  the  preparation  of  a 
rensible  proposal  or  plan  for  educational  dec  is  ion  making  in  the  70.s,  these 
■  telling  judgments  delivered  by  half  the  participants. 
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For  all  intents  and  rniri)Ovr>c  • 

.  ^^^^^^  ^^^^  -  "  -  - 

— ...  •    -  - 

Hits  report  has  to  this  „„i„,  ^ 

,  -  have  given  hints  Of  s™,.   .  r  „      ■  dat.. 

_^  y  our  perceptions  ot  those  dat^£  In^h^ 

'Jd.v..      In  this  section  we  dr-mrh  r 
Co  present  our  ir.          •  -         "  fJt-part  from  our  d.aa 

t     sent  our  impressions  about  sGverainf  M  ■• 

veru.,  of  those  issues  and  to  raise  th.-  I 
question:     "Are  na^^on.l  '^-^isc  the  l;,r'^or  . 

^re  national  summer  instirn(*.c-     i    •  • 

^e.g.,  SIX  weeks)  a  diverse  '.ro.m  of  •  • 

t-o.ic3   <ioo3ei.  tie.  to  s.e  "  ^  "  ^ ^  ^ 

to  some  popular. thome) ,   at  all    f       -.i  • 
these  times?  >  1   fc-as ible  in 

-^^^Slinis^rat_ive  Staff 

feel  confident  that  we  can  nff 

can  offer  .'iome  aporonrist^ 
characteristics  •     .  '    '  ^■";?8e«t ions  about  . 

^     -tics  required  of  the  ad.misc  rat  iv.  .caff  if  .   ,      ,      ^  -., 

-        '""'"'"^^'^^  ^^^^^-^^^^  it  is  i„  ■ 

individual  who  de. •  .  ^u-porativ.  that  tK. 

V  Who  designs  any  such  institute  actu.n 

..      .  ,,.  ucc  actu^aiy  direct   it.     rJor  nn^         '  . 

'-:-V»u.,t,.e 

:    ti»e  and  effort  i      >.  ""'tute.  he  n^st  ai.o  in.est  con.iderahU- 

effort  «  the  months  prior  to  the  .tart  of  the  inatit^ee  u 
unlikely  that        4       .  i-^Lute.     It  seems 

.  r  tH«  an  institute  director  can  he  effective  so  Ion-  ,s  ^ c  ^ 

".•W  :insrit.,ion^..    ro  .o  an:effective  ,oh'  f         .  -  : 

:«rector-.st  he  free  .:        H         ^      ^  ' -  'natitut.,:  the  ' 
■  ""n.i-ents  and  re.ponsihiiiti.s  „f  ,  . 

/■ay-to-day  duties.     As  Ion,  as  the  di r.c  tor  re- i  '     v  '   ^  re«uUr        .  , 

^  A     cor  regains  nhvs  in  n  w  . 
horn,  institution^  hVMll  fi  H  -  ^        -  rV'^icall/^  Proxi^iate  to  hi.  Vi  .  ^  _ 
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Sudden  emergencies,  unexpected  complications  occur  too  f requently-particularly 
in  a  university  setting-for  us  tc     a  able  to  assume  that  a  "paper  cormnitment" 
to  an  institute  will  in  fact  hold  up  on  an  assumed  100%  basis. 

The  task  of  the  administrative  staff  of  such  an  institute  is  enonnaus.' 
In  fact,  its  very  enormity  may  dictate  that  such  institutes  be  abandoned. 
Logistics,  timely  funding,  communication,  facilitation,  interpersonal  skills, 
administrative  kr.ow-how  are  all  esse..tial.    To  put  together  a  staff  possessing 
all  the  necessary  skills  is  in  itself  a  complex  task.     Is  it  worth  it? 


Consultants 


I  can^t  believe  your  song  is  gone  so  soon. 
I  barely  learned  the  tune 
So  soon 
So  soon. 

^    . '  ,  ■  ■      -  ' 

--Paul  Simon,  "So  Long,  Frank  Lloyd  Wright" 

The  "parade  of  stars"  approach  to  a  summer  institute  may  be  ego-satisfyins  to 
some  institute  directors .    .It  may  meet  the  needs  of  a  minority  of  the  participants 
in  the  institute.     But  we  seriously  question  its  potential  for  upgrading  the 
skills  of  participants  on  any  lasting  basis.     To  he  exposed  to,   to  r...t,  to 
acquainted  with  the  Cooks ,  Walbessers ,  Stuff lebeams ,  et  al . ,  is  well  and  good 
but  it  does  not  necessarily  -erve  as  an  effective  or  efficient  training  strategy. 
.Institutes  organized  around  this  concept  compound  the  problems  by  intro^ 

the  need  for  coordination,  transition;  and  organization  to  an  almost 
degree.    The  demands  such  an  approach  places^,<^ 
;ipa»zts  ,are^.hard..tomeet..:  ^  I^;  is  :  even,,.barder  to^vjus^^^^^^^^^^ 
demands  in  the  first  place.     Is  it  worth  it?  : 


0  ■ 
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Time, 
Time , 

Time,  see  what 's  become  of  m<2 

While  I  loolied  around  for  my  possibilities. 

I  was  so  hard  Co  please. 

Look  around, 

Leaves  are  brown. 

And  the  sky  is  a  hazy  shade  of  winter. 

--Paul  Simon,  "A  Hazy  Shade  of  Winter" 
It  is  time  to  quit  kidding  ourselves.     The  almost -invariable  lateness  of  funding 
simply  does  .not  allow  for  any  reasonable  recruitment  and  selection  procedures 
to  operate.     Grandiose  schemes,  elaborate  critek  are  just  so  much  verbiage. 
The  notion  that  a  sunnner  institute  can  serve  the  needs  and  desires  of  a  cross- 
section  of  America  is  also  unrealis  tic .     An  alternative  to  be  considered  is  a 
series  of  narrowly  focused,  short-  or  long-tern,  institutes  or  workshops,  each 
serving  '  homogeneous  group  of  participants-homog..eous  with  respect  to  job 
responsibiUties,  educational  backgrounds,  perhaps  even  regional  locations.  Is 
it^  worth  it? 

Participant  Obsr^rvprc;  rind  the  ;  -    ^    \r  - 


If  some  naive  soul  insists  on  trying  to  conduct  another  institute  which 
is  organised  around  the  concept  of  bringing  in  "expert"  consultants  for  relatively 
Short  periods  of  time  to:  impart  their  wisdom  and  know^^^ 

group  of  participants,  he  should  be  required  to  build  into  his  institute  an    ;  ' 
^eUborate^^;and:det^ 

f>bservers,::aware:and;,^ 

feeclba^k  techniques  are  all  essential.  :  0 
Institute;  was  ;thU^ 

vitaUy  necessary,  especia^^ 
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involved.     Once  again,  the  indication  is  that  ve  might  be  better  off  abandoning 
this  concept  of  institutes  than  investing  resources  in  efforts  to  make  them  work. 
Is  it  wortlj  i,t1     This  question  can  be  answered  for  the  New  Hampshire  Institute: 
Without  a  monitoring  component,  it  is  very  doubtful  that  the  Institute  would  have 
succeeded  at  all,  * 

Evaluating  an  Institute       .  . 

I  can^t  say  just  what  is  real,  all  I  know  is  what  I  feel 
I  know 

The  time  it  is  today,  and  we  must,  find  our  way 

--Russ  Giguere,  '*The  Time  It  Is  Today" 
Earlier  in  this  report  we  described  th^?  procedures  and  instruments  we  used  in 
our  attempts  to  evaluate  the  New  Hampshire  Ins titute .     We  are  now  going  to  try 
to  "evaluate  the  evaluation**'    We  address  ourselves  to  the  question:     Wliat  worked 
well  for  what  purposes? 

Participant  observers ,  while  sometimes  invaluable  to  the  administrative 
staff  of  the  New  Hampshire  Institute,  turned  out  not  to  be  especial'ly  useful  to 
us  in  our  evaluation.     Perhaps  we  simply  did  not  know  enough  about  how  to  make 
use  of-  then.  ■  Perhaps   tihe  ir  ::  ri:::ary  all.?-iaricc  to  the  In::;i:  i  Lute  dir^.ccor  *:ad  ' 
some  effect.     Perhaps  the  use  of  participant  observers  as  the  "eyes  and  ears"  of 
external  evaluators  is  inappropriate.     I'Aiatever  the  reasons,  we  cannot  justify 
tlm  inclusion  of  participant  observers  whose  primary  duties  are  to  serve  the 
external  evaluation.     If  we  were  running  an  institute  (heaven  foiliO,d!)  ,  we  would 
insist  on  having  "our"  participant  observers  included.    If  we  we. :  evaluating 
that  institute,  we  might  say,  "Participant  observers  are  good  to  have  around.  We 
would  like  to  talk  to  them  now  and  then.     We  would  like  to  see  any  reports  they  ■ 
present .     But  we,  can  get  along  without : them. "    One  participant  observer  at  the 


New  Hampshire  Institute  did  provide  a  valuable  summary  document  dealing  with  some 
of  his  more  ^'personal*'  observations  of  the  participants  and  the  impact  of  them 
on  the  Institute  and  the  Institute  on  them. 

On  the  other  hand,  our  experience  with  the  New  Hampshire  Ins titute 
gives  us  no  basis  for  totally  re j ec t ing  the  appropriateness  of  participant 
observers  in  the  evaluation  process.     If  they  were  selected  from  permanent  members 
of  the  evaluation  staff,   they  might  prove  extremely  valuable.     We  do  not  know. 
We  cannot  say.     We  would  like  to  find  out. 

Weekly  Participant  Opinionnaires  were  quite  useful  as /the  Institute 
progressed      If  we  had  it  to  do  over,  however,  we  would  not  use  the  same  form 
week  after  week.     Participants  tended  to  respond  "automatically'*  in  later  weeks 
simply  because  they  had  become  accustomed  to  the  common  forcu     Their  utility  was 
limited,  however,  insofar  as  an  overall  evaluation  is  concerned.     To  obtain  maxi- 
mum   ,.Jue  from  there  instruments  requires  that  they  be  dealt  with  almost  immedi- 
ately following  their  administration.     As  formative-evaluation  instruments,  their 
function  was  to  provide  the  Institute  staff  with  feedback  about  problems  as  they 
occurred  while  there  was  still  time  to  correct  these  T?robleTrs  b'?fore  t:bpv  boc^""/^ 
serious.     During  the  six  weeks  of  the  New  Hampshire  Institute  some  of  the  potential 
utility  of  these  instruments  was  lost  because  of  the  one-week  to  ten-day  delay 
between  the  time  participants  responded  to  the  questionnaires  and  the  time  the 
evaluators  could  provide  an  analysis  and  summary  of  those  responses  for  the 
Institute  staff.     Summary  Opiniormaires ,  in  contrast ,  provided  a  wealth  of  data 
for  the  evaluators. 


Site  visits ,  during:  which  evaluators 


observed,  listened,  and  interviewed. 


were  an  integral  part  of  the  evaluation.     They  were  a  .  almost-satisfactory  sub- 
stitute for  continuous  observation  and  monitoring  of  the  Institute  for  \rposes 
ERIC  an  external,  summative; evaluation^ 
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it  worth  it?     AS  one  participant  conmiented  on  the  Summary  Opinidnnaire, 
"Institutes  such  as  this  are  representative  of  an  old  world  that  is  rapidly  decay- 
in.^."    Such  dissatisfaction  with  the  Hew  Hampshire  Institute  was  not  typical  of 
the' participants,  but  it  does  represent  a  viewpoint  with  which  the  evaluators 
(and  we  suspect  the  Institute  staff  as  well)  are  quite  sympathetic.  Throughout 
this  final  section  we  have  been  asking.  "Is  it  worth  it?"    Our  considered  answer  ' 

is  "No."    We  have  suggested  one  alternative,  a  series  of  narrowly  focused,  short-  

or  long-term  institutes  or  workshops ,  each  serving  a  relatively  homogeneous  group 
of  participants.     There  are  others  we  have  not  considered.     As  we  began  to  enumerate 
the  factors  required  for  institutes  of  this  ilk  tc  succeed,  as  we  began  to  ponder 
the  magnitude  of  the  investment  in  time,  personnel,  and  money  required  for  success, 
we  found  ourselves  saying,  "There  must  be  a  better  way!"  , 
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CIRCE  Attitude  Scale  1,4b 


Name. 


Different  people  have  different  ideas  about  the  ex'aiuation  of  educational  programs. 
Some  believe  chat  maintaining  a  good  school  and  improving  instruction  require 
carefully  planned  evaluation:   Others  believe  that  evaluation  activities  interfere 
widi  teaching  and  learnings  doing  more  harm  than  good. 

Differ<int  people  see  different  purposes  for  educational  evaluation.  Certain  people 
arc  orF~r«ced  mcce  to  pupil  behaviors  or  to  classroom  conditions  or  to  other  aspects 
of  the  program. 

Responses  CO  the  items  on  this  attitude  scale  provide  us  with  6  scale  scores. 
T)t*'h en  plotted  on  the  profile  sheet  below  they  are  expected  to  indicate  the 
/cspondenr's  attitudes  toward  educational  evaluation. 


Directions  for  Self  Scoring 

Start  in  the  oppo:iite  comer  of  thi 
page.  For  each  scale  check  youi 
sheet  to  see  ho w  you  responded  s 
each  of  the  eleven  items.  For 
Example,  with  SCALK  V  how  did 
you  mark  Item  f2?  If  you  marked 
it  * *A"  put  a  check  In  die  paren- 
theses. Put  die  number  of  check 
in  the  box.  Mark  each  horizontal 
scale  (at  the  right)  at  til e  number, 
point  shown  in  its  box.  Draw  vol 
profile  by  connecting  your  scores 
on  the  five  scales.  1-V.  Then  fift 
your  CONFIDENCE  score. 
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I.  A  RESEARCH  orientation  to  Evaluauon 


012    3  4 
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The  person  high  on  this  scale  appears  to  believe  that  evaluation  shoui 
rely  on  precise  measurement  and  statistical  analysis  to  gain  general 
understanding  of  why  programs  do  or  do  not  succeed. 
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II.   A- SERVICE  orientation  CO  Evaluation     oi  2   3  4 


7    8  9  » 


The  person  high  on  this  scale  appeiirs  to  believe  that  evaluation  shouI 
be  designed  according  to  the  needs  of  ihe  educatofs  involved  so  as  to 
aid  them  in  their  present  work  and  future  decisions.  - 
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jtll.   A  TEACHING  orientation  to  Evaluation  oi  2   3    4       5       6      7    8  9 


Ttie  person  high  on  dii^;  scale  appears  to  believe  tliac  evaluation  shoo  I" 
be  focused  considerably  on  the  quality  of  teaching  and  should  discover 
the  intrinsic  merit  in  facilities  and  in  instruction. 
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IV.   OBJECTIVES  orientation  to  Evaluation  o  1  2   3  4 
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therefore  evaluation,  should  be  focused  considerably  on  .irriori  .st.ite-' 
ments  of  objedruves,  that  die  merit  of  the  program  is  iargeiy  indicntcd 
by  the  success  of  students  in  reaching  those  objectives. 
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V.   A  JUDGMENT  orientation  to  Evaluation  012  3  4 


7    8  9  > 


The  person  high  on  this  scale  appears  to  believe  that  educational .  "  ; 
evaluation  is  largely  a  matter  of  establishing  the  worth  of  the  prGcrain 
for  various  purposes  as  perceived  by  various  groups  of  persons  in 
and  around  the  p'rogram. 


OS 

< 


To  obtain; an; overall..  V......CONFIDENCHJN  EVALUATION  score, 

do  the  same  thing  with 


the.  check -list  at  die  right. 


012   3  4 


7    3  9  X)TI 


4^  ^  ^ 
>  >  >: 


»u  ^  v»  >J 


CIRCE  Attitude  Scaie  No.  1.4  X 

Attitudes  fowcifd  EducQtionol  Evaluation.,  Heio^'  ate  a  nuchcr  of  M.ucMK'nts  about  th.  cvalu.uion  oi  cducatji^n..!  r-o.-aPus 

pro^ra.n.  can  be  a  lesson,  a  course,  a  '.lioie  cur.'iculu.v,  or  ;,nv  :r.uni,M,i;  actlvuv.   C:u„si-aer  each  M.uenicn,       a  •^VunnV..', 
of  optn.on.  If  yo„  a^rce  at  least  a  little  hit  with  the  su:cmv:n\  circk-  the  letter  .1.   If  v«u  J....a,ree  cv.„  .  i;ttl,  h. 
the  statement,  c.rcie  the  letter  D.   If  vou  both  a..ree-  and  disagree,  or  if  v..  h.tve  no  opinio,,,  leave  the  Icctc';.  UncircU-i, 

A  -AGREE  D  -  DISAGREE  Blank  Neither 

1.  A    D    The  rnaior  purp^^se  of  an  educational  evalu.itioti  .stady  should  be  to  ^yather  information  that  sviii  h6'  l.elptu!  to 

the  educators.  w     ^  t  ^" 

\. 

2.  A    D    It  is  important  for  the  pro_«rat,i  evalu.itor  to  find  out  ho-,v  u-ell  variou.s  people  like  the  program. 

3.  A  ■  D    Generally  speaking,  an  educational  program  should  be  evaluated  with  reference  to  one  or  niorc  "C"iurol-' 

programs.  '  . 

4.  A    D    The  evaluator  should  accept  the  responsibility  of  finding  the  .strongest,  mosc'defensible,  and  publ'dv  attrac- 
.    '.  tive  points  of  the  program.  t"  .' 

5.  A    D    In  evalu.atin£  a  program,  it  is  at  least  as  important  to  smdv  and  report  on  the  t%'pes  of,  teaching  .5  it  is  rn 

study  and  report  on  the  amount  of  learning. 

.     6.    A    D    The  evaluator  should  draw  a  conclusion  as  to  whether  or  not  the  goals  of  the  program  are  worthwiiHe.        .  ' 

7.  A    D    It  is  more  important  to  evaluate  a  program  in  comparison  to  what  other  programs  do  than  to  eva^^^  it  ^ith 

reference  to  what  Its  objectives  say  it  should  do. 

8.  A    D    Principals  and  superintendents  should  not  gather  data  about  the  quality  of  instruction  in  the  classroom. 

9..    A    D    TJie  task  of  putting  educational  objectives  into  writing  is  more  the  responsibility  of  the  evaluatof  than  that 
or  the  educator.  a  .i 

10.  A    D    It  is  essential  that  the  full  array  of  educational  objectives  be  stated  before  the  program  begins.  - 

11.  A    D    Evaluatiot|  studies  would  improve  if  thev  gathered  more  kinds  of  information,  even  if  at  the  expense  of 

gathering  less  reliable  information. 

12.  A  D  Evaluators  should  ignore  data  that  cannot  be  objectively  verified. 

13.  A  ,D  Education  should  have  more  of  an  engineering  orientation  than  it  now  has. 

.14.  A  D  The  job  of  an  evakiator  is  raostlv  o.Te  of  firidin  e  c\;:  :;oT»-  ;ve!!  s:-\^  'V'-  'ea:  -  ■  -  \-  .  ,  : 

.15.  A  D  Evaluation  should  aid  an  educator  in  revising  his  goals  even  while  the  program  is  in  progress. 

.16.,    A    D  .  The  process  of  decision-making  about  the  curriculum  is  one  of  the  weakest  links  in  the  pre.senc  of  ^^ation  of 
tne  Schools.  ,       .  - 

17.  A    D    Educators  have  some  important  alms  that  cannot  be  stated  adequately  by  anyone  in  cerms  of  szuie^^  behaviors. 

18.  -A  .  D    Information  from  an  e valuation. study  is  not  worth  die  trouble  it  makes. 

19.  -  .  A    D    The  first  job  in  instniction  is.the  formulation  of  a  statement  of  objectives,    ,  : 

20.  A    D    A  teacher  should  tell  his  students  any  and  all  of  his  teaching  objectives, 

-il.  ^^  ■  ^'he  maior  purpose  of  educational  evaluation  is  to  find  ou  :  . 

22.  ;  A    D    The  evaluator  should  be  a  facilitator  more  than  a  critic  or  reformer  or  scholar/ ^ 

23.  A    D    Some  school  experiences  are  desirable  because  they  round. out  n  child^s  life^whether  or  not  thev  Ir^^rcise 

his  competence  or  change  his  attitudes. 

,  Please  turn  over 


24.  A    D    An  evdiuator  should  find  t)ur  if  the  ccnchint;  is  in  fact  die  kind  due  the  school  faculty  expects  it  to  be. 

25.  A    D    Whether  or  not  an  evaluation  report  i.s  any  cond  should  be  decided  prettv  much  on  the  same  grounds  that 

research  journal  editors  use  to  decide  whether  or  not  a  manuscript  should  be  published. 

26.  A    D    Ihe  main  purpose  of  e\Tiluation  i.s  to  £;ain  understanding  of  die  causes  ot  ^^ood  irrstructlon. 

27.  A    D    Desciiption  and  value  judgment  are  equally  important  components  of  evaluation. 

28.  A    D    In  conducting  an  evaluation,  there  is  no  justification  for  the  exercise  of  subjective  judgment  of  any  kind 

by  the  cvaluator. 

29.  A    D    Educational  evaluation  is  a  necessary  step  in  the  everyday  operation  of  the  school. 

30.  A    D    The  strategy  of  evaluation  should  be  chosen  primarily  in  terms  of  the  particular  needs  the  sponsors  have 

for  evaluation  data.  -  . 

JL    A    D    The  educational  evaluator  shou'd  attempt  to  conceal-all  of  his  personal  judgment  of  the  .worth  of  the 
program  he  is  evaluating. 

32.  A    D    The  sponsor  of  an  evaluation  should  have  the  final  say-so  in  choosing  or  eliminating  variables  to  be 

studied. 

33.  A    D    The  main  purpose  of  educational  evaluation' is  to  find  out  what  methods  of  instruction  work  for  different 

learning  situations. 

34.  A    D    Parents':  attitudes  should  be  measured  as  part  of  the  evaluation  of  schoo  1  programs. 

35.  A    D    An  evaluator  finds  it  almost  impossible  to  do  his  job  without  intruding  upon  the  operation  of  the  program 

at  least  a  little. 

36.  A    D    All  important  educational  aims  can  be  expressed  in  terms  of  student  behaviors. 

37.  A    D    Some  educational  goals  are  best  expressed  in  terms  of  teacher  behaviors. 

38.  A    D     It  is  essential  that  evaluation  studies  be  designed  so  that  the  findings  are  gencralizable  to  other  curricula. 

39.  A    D    An  evaluation  study  should  pay  less  attention  to  the  statistical  significance  of  a  finding  dian  an 

instructional  research  study  would. 


le  evaluation  planning  can  1  e  done  before  you  get  a  statement  or  instructional  objectives. 


of  stated  educational  goals.    ,  : 

•44.    A    D    An  evaluation  of  an  educational  program  should  include  a  critical  analysis  of  the  value  of  the  goals  of 
the  program. 

A    D    Every  teacher  should  have  formal  ways  of  gathering  information  about  die  strengths  and  shortcomings 
of  his  instructional  program.  *  * 


40. 

A 

t) 

•il. 

A 

D 

A 

D 

43. 

A 

D 

46. 

■■A: 

D 

47.'/. 

A 

D 

48. 

A 

D 

If  an  evaluation  study  is  well  designed,  the'primary  findings  .}re  likely  to  improve  decisions  made  by 
administrators,  teachers,  and  students  themselves. 

49.  .  A  .  D    When  the  evaluator  has  to  choose  between  helping  this  staff  run  its  program  better  and  helping  educators 
everywhere  understand  all  programs  a  little  better  he  should  choose  the  latter. 


INSTITUTE  PARTICIPANT  OPINIONNAIRE 


CIRCE" 
"SumiDer  1970 


5-DIGIT  CODE 

DIRECTIONS:    The  following  items  have  been  written  to  enable  you.  to  say  how  you 
feel  about  several  aspects  of  an  instructional  we^k  of  your  Institute.    Check  one 
response  category  for  each  question  and  comment  if  you  wish  to  clarify  your 
response. 

Select  any  5-digit  code  number  and  use  it  throughout  the  Institute  on  these  opin- 
ionnaires . 


1.    Are  you  enjoying  yourself  at  this 
Institute? 


Yes     Yes- ''but"       No    1  don't  kn-.^w 

□  □  □  □ 


Coinment ; 


2.    Are  you  -getting  the  chance  to  talk 
to  the  staff  as  muc'it  as  you  would 
like  to? 


□  □  □  □ 


CotniTient ; 


3.     In  your  opinion,  is  the  entire 
Institute  well-organized? 


□  □  □  □ 


Coinment : 


4. 


Has  the  general  administration  of 
this  past  week  been  well-organized? 


□  □  □ .  □ 

Comment: 


a/ 

4^ 


Yes      Yes-**but"       No-    1  don't  know 


5.    Have  :'0u  been  getting  along  well  with       j               i               f    1         j  j 
other  participants  at  the  Institute?         '   ^  '  ' — ^  ' 


Comment : 


6.    Did  you  enjoy  the  instructional  ^taff       j     1          FH          \    1          F  I 
for  this  week?  '  '  ' — J  1— J  I  1 


Comment : 


?•     Do  you  feel  you  learned  a  lot  this 
week? 


Comiftent : 


Corninsnt : 


□  □  □  □ 


8,     Is  the  Ir^stitute  paying  enough                  j     I          j     1          [    j  f'~| 
attention  to  your  important;  problens?       ^ — I  L_j  |  }  }  I 


Would  your  profe.^sionai  peers  back  f    j  I     j  ["1  FH 

home  be  interested  in  what  you  '  '     '    1  ^  LJ  t  I 

learned  this  week? 


wOiTiment : 


10.    Do  you  think  you  could  teach  another        [~1  j  1 

group  what  you  have  learned  this  week?      L_!  |  J  \  ] 


Coiranent : 


-3- 


ERIC 


11.    Is  the  Institute  meeting  your 
personal  expectations  for  it? 


Yes     Yes- •'but"       No    I  don't  know 

□  ,□•  □  □ 

Coffimentj 


12.  Have  the  staff  failed  to  change 
or  correct  certain  matters  that 
were  brought  to  their  attention? 


□  □  □  □ 


Comment : 


13.    Are  you  getting  the  chance  to  study 
as  much  as  you  would  like  to? 


14.     Are  you  pefrtinc  suf.^icient  oppor- 
t':^nity  to  socialize  ar;d  recreate? 


□  □  □ 


CoiTiinent : 


□ 


Connent; 


□  □ 


15,    Was  the  contend:  for  thi.s  week 


chal^ 


engine;  ir:port;?nt? 


□    □    O  0 


Coiuirient ; 


16 r    Are  you  working  hard  crc.irh? 


Comment: 


Did  this  week  '^ang.  together''  as 
an  instructional  whole  for  you? 


Yes      Yeg.^ut^'       No    I  don't  know 

□  □  □  □ 


Comment: 


Would  you  want  to  teach  another 
group  what  you  have  learned  this" 
week? 


□  □  □  □ 


CoiBnient : 


Should  the  presentation  for  this 
week  have  been  preceded  with  more 
readings,  discussion,  background 
materials? 


□    □    □  Q] 


Comment: 


Would  you  reccirriend  including  all 
or  part  of  this  wnch  in  a  pcckage 
to  be  used  in  summer  institutes 
elsewhere? 


L-j  □  n  D 


CoTTTiient ; 


If  70U  have  a  message  fcir  ' the  people  running  this  Institute ,  please  write  it 


SUllMARY  OPINIONNAIRE 


if 


CIRCE 
Summer  1970 


.5-DIGIT  CODE 

DIRECTIONS:     Throughout  this  Institute  you  have  told  us  how  you  feel  about  a 
number  of  aspects  of  the  instructional  program  of  this  Institute.    Now  we  are 
asking  for  some  OVEP^L  reactions.     Check  one  response 'category  for  each  ite" 
and  comment  if  you  wish  to  clarify  your  response.  ' 

Please  use  the  san>e  5-dlgit  code  on  this  Opinionnair  -  that  you  have  been  using 
throughout  the  Institute.  u&ing 

1.    To  what  extent  do  you  feel  the  following  objectives  were  achieved:  ; 

Very      Fairly      Mini-  Not 
Well       Well       aially    At  All 

a.  An  understanding,  of  the  problems  inherent 
in  the  management  of  an  educational  eval- 

uation  project.  g         ^  g  ^ 

b.  A  knowledge  of  methods  and  procedures  for 
evaluating  those  curriculum  changes  with 

which  you  as  an  Institute  participant  are 
concerned. 

C.    An  awareness  of  the  major  alternatives 

^v^il^ble  in  terras  of  educational  research 

.        xcthodoiogi...  c:>  □      e      □  □ 

.  d.    An  understanding  of  the:  relatipnship  of 
evaluation  to  the  problems  associated 
with  the  development,  dissemination,  and 
adoption  process  in  educational  innova- 
tion. 


□     H     □  •  □ 


□      0      Q  Q 

A  familiarity  with  communications  tech- 
niques applicable  to  proper  implementation 
of  th'e  decision-making  process  at  various 

levels  of  the  educational  system.  [|         111         (j)  Q 


2.    You  probably  had  some  personal  objectives  which  were  not  the  same  as  those  of 
the  Institute/    If  so,  would  you  indicate  below  (briefly,  please)  what  they 
were  and  how  well  they  were  achieved. 

Very     Fairly     Mini-      Not  . 
(MOST  COMMOriLY  OCCURRING  OBJECTIVES)  Well        Well        mally     At  All 


  El    □    □  g 


..   :  :        □    D    □  □ 


□      □       □  □ 


3.    For  each  of  the  four  major  sections  of  this  Institute,  indicate  whether  time 
allocated  was  too  little ,  about  right ,  or  too  much; 

Too       .  About  Too 
Little      Right  Much 


a.    Cook:     One  v:eGk  w?.5^-- 


M       fi^  R 


b.    Walbesser:    Two  weeks  vjas-- 


□     □    ,  S 


c.    Stuff lebeam:    Tx^o  weeks  was-- 


d,    Asher:    One  week  was-- 


□  □ 


ERIC 


4.    For  each  of  the  four  major  sections  of  the  Institute,  indicate  by  checking 
the  appropriate  box  the  extent  to  which  you  think  the  material  presented 
during  that  section  was  relevant  to  your  own  activities  or  concerns: 


Very       Sornewhat    Slightly  Not 
Relevant    Relevant    Relevant  Relevant 


a.    Section  One  (Cook) 


a 


□ 


b.    Section  Two  (Walbesser) 


□ 


□ 


□ 


c.    Section  Three  (Stuff lebeam) 


□ 


t3 


□ 


d.    Section  .Four  (Asher) 


□ 


□ 


□ 


For  the  following  items ,  please  mark  the  appropriate  box,  using  these  alternatives 


SA  =  strongly  agree 
A  =  agree 
U  «  undecided 
D  -  disagree 

SD  =  strongly  disagree 

5.    The  content  of  this  Institute  has  direct  practical 
utility  in  my  institution. 


SA 


12\  R 


D  SD 
□  □ 


■  Coment : 


6.  In  general,  the  approaches  presented  here  are  not  SA  A 
.  ..   feasible  in  my  institution.  foj  fS] 


U        D  SD- 

3  0  Dl 


Cotment : 


-4- 


4^ 


7.    The  benefit  I  derived  from  the  Institute  was  not 
worth  my  expenditure  of  time. 


Comment ; 


SA 


□  □  □ 


D  SD 


IC 


0 


8. 


This  Institute  compared  favorably,  overall,  with 
other  institutes  I  have  attended. 


Comment : 


Does 
Not 

SA     A     U      D    SD  Apply 

□  B  □  □  □  B 


9.    The  diversity  in  content  and  approach  presented 
by  the  consultants  was: 

a.  Appropriate  to  the  objectives  of  the  Institute. 
Comment: 


SA       A       U  D 

13  EE 


SD 

m 


ERIC 


b.  Satisfactory  to  me  personally. 
Comment   „  - 


SA 


U       D  SD 


B  d  a  s  □ 


c.    Too  great  for  me  to  be  able  to  assimilate  and 
integrate  things  as  well:  as  I  would  have  liked. 


Coiranent : 


SA       A       U        D  SD 

H  0  0  B  H 


10.     If  this  Institute  were  to  be  offered  again,  I  would  SA 
recotTiraend  to  others  like  myself  that  they  attend  it,  fsj 


11. 


Comment : 


Consultants  were  generally  not  available  for 
questions  and  discussion  outside  of  the  times 
scheduled  for  formal  presentations. 


Comment : 


D  SD 


□  □  □ 


SA       A       U       D  SD 

□  □□(HQ 


12.    After  participating  in  the  Institute,  I  feel  more 
competent: 

a.  To  approach  and  conduct  evaluation  studies 

b.  To  design  and  carry  out  research  projects 


SA 


A       U       D  SD 

-  E  [0 


18 


SA       A       U       D  SD 

El  E  CD  H  m 


c.    To  develop  and  implement  sound  decision- 
making strategies. 


SA       A       U       D  SD 

E  d  [D  0  0 


ERIC 
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13.    For  each  of  the  following  descriptive  statements,  "indicate  by  checking  the 
appropriate  box(^s)  whic!ii  of  the  sections  of  the  Institute  it  applies  to. 
The  following  code  is  used  for  the  column  headings: 

(1)  ^  Project  management  models  and  techniques  (Cook) 

(2)  =  Behavioral-  Objectives,  design  of  assessment  tasks,  and  construction 

of  learning  sequences  (Walbesser  et:  al.) 

(3)  =  Evaluation  models  and  techniqueiS,  seminars  on  problematic  simula- 

tions (Stuff  lebeam,  Cuba,  and  Hanmiond) 

(4)  =  Problems  associated  with  developing,  disseminating,  and  adopting 

educational  innovations  (Asher  and  Heisley) 


a. 

This  material  could  have  considerable 

(1) 

(2) 

(3) 

(^) 

applicability  to       work  at  my  own 
institution. 

17 

16 

23 

14 

bo 

I  really  don't  see  how  this  could  apply  to 
the  problems  with  which  I  have  to  work. 

3 

4 

2 

4 

c . 

It^s  likely  that  I  will  be  able  to  put 
this  to  good  Use. 

18 

18 

24 

18 

d. 

I  don't  see  how  this  material  could  be  included 
in  another  Institute  in  the  absence  of  the 
consultant (s)  who  presented  it  here. 

3 

4 

10 

2 

e , 

This  material  Would  be  readily  transferrable 
to  other.  Institutes  as  an  instructional 

■  packri.^.e,  without  ^^s  the.ccn?ult- 
ant(s)  who  presented  it  here  be  involved. 

16 

7 

16 

I  would  recommend  that  my  colleagues  attend 
a  Workshop  0^  Institute  which  had  this  topic 
as  its  centJ^al  focus. 

13 

12 

26 

12 

the  Institute^ 

(Please  check  the  appropriate  response.) 


Facilities  . 

Time  schedule 

Budget 

Format 

Materials 

Equipment 

Staff  (Al  EXwell,  Mo  Oliver,  Ev  Barnes) 
Consultants 

Participant  Observers 

(Charles  Williamson,  Stuart  Pickard) 

Participants 
Food 


lent 

,  O.K.      dpn'HpH    oq.|-,«-f,  pnor 

12 

15 

0 

20 

2 

2 

;12  ^ 

6 

3 

1 

■■  3 

11 

8 

;  ;:3, 

■  ■.  3" 

14 

11 

0 

14 

1 

12 

y  9:; 

■■■\\ 

;  13 

.  ...  ^4  ; 

0 

9 

. .  ■ — 

6 

8 

3 

2 

13 

5  ' 

0 

-•1 

9 

0 

2  ■ 

;erjc 
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15/^:  W^^       of  ;the:  following  best  describes  your  overall  reaction  to  the  Institute? 


14 


Very         Fa^'orable      Undecided    Unfavorable  Very  : 

Favorable  Unfavorable 

16.    A  six-week  time  allocation  for  an  Institute  like  this  one  is: 


Much  too 
long 


A  little 
too  long 


About 
right 


A  little 
too  short 


Huch  too 
short 


17, 


Do  you  have  any  message  for  the  staff,  consultants,  or  evaluators  of  this 
Institute? 


ERIC 


CIRCE  AlMu^ie  S, tl.j.  1,4 


A  AGREE 


.A     !'3    it  is-.. 


<.i'f  Lfif  cdiic.iLor. 


A 

[.) 

.A 

1) 

I) 

1.1  ; 

A 

i) 

;\ 

i) 

13. 

A 

I) 

IK 

A 

.  i) 

A 

I) 

.A 

I) 

A 

D 

A 

D 

A 

D 

'a 

i) 

■  A 

D 

A 

D 

fc 

.  A 

D 

^Mn(:.ir.ai:>  ano  ;.\i|HrruHeiiaci- 1;.' ^hou ki  ;;or  leather  .i.ira  :iU,u:rd)c  ;-.:a;i:v       u'ls ui;.:u.a;  -lu 


idtci  before' T;.;;  p-r.  'j^raii)  i  cia:;  • 


:uuJjc;.  would  iiupi.r^,::  i i"  il:-y  gathcTcJ  i\)orc^  ki::As.ri  :::^>r^pati^:a^  ovci!  if  at  liK-  JAPn-^^-^  : 
^.itiiCiUit^  less  ri'li,.ihi<^  :n  I'tUMKU^ioii ,  ,      ■       r'    ^    '  •  p.  ' 

r-valuators  sl^oiiLi  i^nure  liaM  char  cannot  ito  wi;j;^cn  vc  ly  vcJificii.  '  ,  .■  ■ 

J;aiu(::a(  :un  :>nou[..l  }ia^■''  nxoic  o[.in  t;r-j|^M]t."c.;rrn  u  oricaKatio;)  li'faa  ;c  aaw  h,K'  :' 

Ji^^'  }v.'i-^v.>t  ar5  evaluator  is  ^nosiW  ,,ijc  ai'  fladau:  ciu  hv>\v  .-c  j  J<.:n      liirnri-  vaiat  U.cv  arc  :a,:[]-^^r- ^*  r.-'^  ■k^i-'. 

i'.Viiluati-)ii  ^iioulvl  aid  an  educator  in  revi^dii^^  his  ^;ua:.h  cv^/n  tla:-  pAi^rain  is  m  proi^re.-^s,  ■  '  ■. 

■jhc  ;)ro.:c'Nsa)f^d(;cisiun^tiniicin;>  aUuit  [-he  cu rr; va.i  1  uni  i:-,  one  of  the  wfai-ic^;  links  in  uie  prcscnc  w ;-=^-at I- n  ' 
tlif.'.  8ci io^>ls .  . 

.  r-ducaiof:.  have  some  iniporcant  .uiiis  that  cannot  be  stated  adequately  by  ^inyone  in  terms  of  ►aucic;-^  lud^a 

Infonaatian  irom  an  eva!n:i:u:.n  -audy  is  noi  woi-ih  the  ri'<'uble  K  niakc;. . 

:     ■  . 

'jlif  iirsc  ivtl",  in  irjsiruotion  ir.  du:  [on iiu ! at it..ia  of  a  stntei^jcnt'  ^-i  oldecd'.'e'^- 


1 ! )  a  i  *ja  £  the  w  o  i  l  h  of  vd  au  Is  1 1 ,  i  ] '  j  'O  n  i  n  p .. 


hi-s  C'oaU'i.' fence      clmiiire  ])];,  aiiinadcn 


ey  iuniid  i,>\Lt     (:hild'>;  li      ^vTJcdiC•■  or  uot  d)Cv  i^'^' tea 


2^. 

A 

n 

.A 

.■\ 

r> 
i .' 

I.) 

28. 

A 

1.) 

29. 

A 

D 

:^o. 

A., 

D 

31. 

A 

D 

A 

L) 

A 

/I 

34. 

.A 

D 

35. 

A 

13 

36. 

■  A 

D 

37. 

A 

D 

3S. 

A 

D 

39. 

A 

D 

40. 

A 

D 

41. 

A' 

D- 

42. 

A 

13 

43. 

A 

D 

44. 

A 

D 

A 

D 

46. 

A 

D 

47. 

A 

D 

48. 

A 

D 

49. 

A 

D 

1  he  main  .^...r;>ose       c  valuaui-'-n  is  co  ;;-.iir.i  ujiJcr  s      .iin    *  >r  {lie  c.-^u-c;.       .^wv^.i  iii  struc  i  . 
DcNCfipcioti -and  value  jaJ^inc-ju  arc  c^uai  ly- inipt)r(.ai:t  coinponeius  of  cvaiuacixM^ ! 

h)  conduciing  an  eva  1  aa tli^iu  there  is  no  ; us ti ti ca ti O!)  for  the  cxcrci^^e  oi  subjective  iud_eineii-  .A  .tnv  kia  =  i 
pv  the  e  Vsiluator. 


inc  :.[rarOt^y  c>i  evaiiianon  sUoiiUI  i>c  chose::  pninaniy  in  rcnns  or  tise  parii^:uiar  needs  ^.. 
for  evaluacion  data. 

ihe  edacaciojial  L-vahiaior  siiould  atien^pt  to  conux'.u  all  of  iiis  per.^or.a !  |udi;racnr  oi  the  wortii  v-l  rht 
^ro^erani  he  is  cvahiatirig. 


■studied, 

Tiie  iTiajii  piarpv)..se  of  educational  evaluation  is  to  find  out  what  methods  of  mstructio 
Icnrning  siciuuions. 

Parents'  attitudes  should"  be  measured  as  part  of  the  evaluation  of  schoo  1  programs . 

An  evaluaror  finds  it  ahiiost  impossible  to  do  his  job  without  intruding  upon  .  e  oper 
at  least  a  little. 


instructional  research  study  would, 

40.     A    D     K valuation  interferes  svith  the  running  of  schools  more  than  it  helps. 

Little  evaluation  planning  can  be  done  before  you  get  a  statement  of  instructional  objectives. 

42.     A    13     ']*he  leader  of  an  evaluation  team  should  IrJe  a  teacher. 

The  entire  school  day  and  the  entire  school  experience  should  be  divided  up  and  assigned  to  the  pursuit 
of  stated  educational  goals. 

An  evaluation  of  an  educational  program  should  include  a  critical  analysis  of  the  value  of  the  goals  of 
the  program « 

Every  teacher  should  have  formal  ivays  of  gathering  information  about  die  strengths  and  shortcomings 
of  hi. s  instructional  program. 


ERIC 


administrators,  teachers,  and  students  themselves.      '  \    ■.  . 

\9.  A  D  When  the  evaluator.  has  to  choose  between  helping  this  staff  run  its  program  better  and  helping  educators 
O   .     ,  everywhere  understand  all  programs  a  licilc  better  he  should  choose  the  latter. 


CIRCE    Aftifuds  Scale  lAo 


:\\:ih       chi:' viHt.!  1  cja  i{Mn;: .  i.ii.5:n,,'  in*  ■  ri.*,  rr;' 

Diffcrcni'T^copic  svc  -..liiicrcn:  :-.:;-po->cs  i-^r  <,  J.; i  :io;, ;  r  *.  a! -i.u :  uii .  <' 
arc  oriejucd  inorc  lo,  juipil  beliaviors  or  Cv)  cKissroi;::]  Cvmu.i:  ckmi s 

./.^>por^^>c^i  CO  the 'i  terns,  on  tiiis  ntri  Lucie  sc.iic  p<o\-jJc  us  wi^i^  ;>  scale 
\^:-ca  ]'-wrte-.:  on  (he  profile  .Jicor  (mcLav  tiu;/  .ux  c/xpccccJ  :o  iMai.^aic 
rt='sj)ondenr' s  atti  Ludes- cow  ard  cdu^.td'::5.i !  evaiuar^^n. 


1  v\'.. 
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1.  A  RESEARCH 


< 

U 


^   r-j  I'-j 


G  <  Q  <: 

r'"^!   r*-^.  r^.. 


CO  ':rN 

Tr-^.   rr*i  •'^r 


Ihe  person,  high  oiviras  s.rale  appears  to  beljcva-  ii 
reiy  on  prcciiic  rne;u-;ureiiUMU  arid  .scaci.srlcd  .ia.ii\'s 
understanding  of  wliV  programs  d(.»  or  do  aoi.  succee 


)at  o 


iraia  general 


U 


<  Q  Q 

"^r  'O  <^^^ 


<  O  <  O 

so  00 
.-H  r-j 


pry,   f<^.  rrv 


II.    A  SERVICE  orientation  to  I-lviii  uadon 


012    3  4 


7    a   9  K) 


The  person  hii>h  on  dus-  scale  appear^!  co  believe  djat  evaluacioa  ^i)wa:> 
be  desii;ned  according  to  die  needs  of  the  educators  involved  so  a>;  lo 
aid  them  in  their  j.>resen t  work  and  future  decisior.js. 


11!.  A  TEACHING  orientation  to     valuation  01  2 


3  4 


9  ic 


<  <  <  O 


<  <  < 


rO-  '-^ 

r-i  e4  CO. 


O  <  < 

O  r->  rj 

rr,  XT 


o 
H 


llie  person  high  on  this  scale  appears  to  believe  that  evaluatic>n  biiou! 
be  focused  considerably  on  die  quality  of  teaching  .md  should  discv^vc 
the  intrinsic  merit  in  faciliues  and  in  instruction. 


> 

m 
-J 
< 
u 


o  o  <  < 

On  O 


Q  a     <:     q  < 


r-H 

ro*  *<r  *<r 


o 


IV,   OBJECTIVES  orientation  to  Evaluadon  o  I  2   3    4  *^  5 


9  VI 


The  person  high  on  this  scale  appears  to  believe  dial  instruction,  and 
clierefore  evaiuationvsliould  be  focused  ci^n siderably  on  iipriori  stace- 
ments  of  objectives,  diat  the  medc  of  the  program  is  iarj^ely  indicated 
by  the  success  of  students  in  reaching  diose  objectives. 
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H 

V.    A  JUDGMENT  orientation  to  K valuation   0 1  2   3  A 


7    3  9  x: 


The  person  high  on  this  scale  appears  to  beiieve  diat  educational  ' 
evaluadon  is  largely  a  matter  of  estabii shing  the  worth  of  the  program 
for  various  purposes  as  perceived  by  various  groups  of  persons  in 
and  around  the,.program. 


ft'' 
•I 


< 


To  obtain  an  overall...  

do  tlie  same  diing  with 
the  check -list  at  tlie  nght. 


CONFiDENCE  IN  EVALUATION  score,      cc  Os 

>  >■ 


O  >  ">> 


0  12    3  4 


7     3   9  K)T1 
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INSTITUTE  PARTICIPANT  OPIN^WNASKI 


CIRCE 
Sutnmer  1970 


'  "'"1 

5-DIGlT  CODE 


DIRECTIONS:    The  following  items  have  been  written  to  enable  you  to  say  how  you 
feel  aboi^t  several  aspects  of  an  instructional  week  of  your  Institute  •    Check  one 
response  category  for  each  question  and  comment  if  you  wish  to  clarify  your 
response . 

Select  any  5*-clisit  code  number  and  use  it  throughout  the  Institute  on  these  opin- 
ionnalres...  ' 


1.    Are  ydu  enjoying  yourself  at  this 
Institute? 


Yes     Yes-"but"       No    I  don't  know 

□  □  □  □ 


Coimaent ; 


2.    Are  you  getting  the  chance  to  talk 
to  the  staff  as  much  as  you  would 
like  to? 


Comnient : 


3.    In  your  opinion/  is  the  entire 
Institute  well-^organlzed? 


E)   □   □  □ 


Comment: 


4. 


Baa  the  general  administration  of 
this  past  week  been  well-organized? 


Fi    n    ri  R 

Comment: 
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Yes      Yes -"but"       No    I  don't  knew 

5.    Have  you  been  getting  along  well  with  {~1  m 

other  participants  at  the  Institute?  '  — '         L~J  1->J 


Comment; 


7.    Do  you  feel  you  learned  a  lot  this 
week? 


8 ,    Is  the  Institute  paying  enough 

attention  to  your  important  problems? 


9.    Would  your  professional  peers  back 
home  be  interested  in  what  you 
learned  this  week? 


10;    Do  you  think  you  could  tea<  i  another 


group  what  you  have  learned 


this  week? 


1 — 1 

□ 

Comment 

□ 

□ 

□ 

ConBoent : 

IZZI 

□ 

□ 

□ 

Conment : 

f"  ''t 

□ 

Comment: 

□ 

□ 

□ 

uoitsnent : 

□ 

c 

ERIC 
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11.    Id  the  Institute  meeting  your 
personal  expectations  for  it? 


Yes      Yes-"but*'       No    1  don't  knovr 

□  □  □  □ 


Goasoent : 


12.  Have  the  staff  failed  to  change 
ot  correct  certain  matters  that 
Wre  brought  to  their  attention? 


□  □  o  □ 


Conenemr: 


CosBDent : 


13.    Are  you  getting  the  chance  to  Study        {    j         Fi         j     j         |  1 
as  much  as  you  would  like  to?  ^  ^^^^^^^^ 


14*    Are  you  getting  sufficient  oppor«- 
tunity  to  social lee  and  recreate? 


□    □   Q  Q 


Cofument: 


15.    Has  the  content  for  thle  week  PI         j     I         f~l  PHj 

challenging  and  important?  L-J         L*-J         LmJ  LmmJ 


Comment : 


16,    Are  you  working  hard  enough?  |    "j         |    j         |  | 


Comment: 


Did  this  week  /*hang  together'*  a' 
an  instructional  v^ole  fot  yo 


Yes      Yes -''but"       No    1  don't  know 

□  □  □  □ 

Comment : 


Would  you  want  to  teach  another 
group  what  you  have  learned  this 
week? 


.oimasant : 


□  □ 


Should  the  presentation  for  this 
week  have  been  preceded  with  more 
readings ,  discussion,  background 
materials? 


□  □  0  □ 


LOssnDE^nt: 


Would  you  recommend  including  all 
part  of  this  week  in  a  package 
to  be  used  in  summer  institutes 
elsewhere? 


D  □:  □ 


Jetir         ^  f^'^  the  people  runaiag  this  Institute,  please  write  it 


22.     Dp  you  have  a  message  for  an  instructor  or  visiting  consultant  who  worked 
in  the  Institute  this  week?    Please  write  it  here. 


23.    If  you  were  restricted  to  a  sentence  or  tvo,  how  would  you  describe  the  main 
idea  of  this  past  week? 


24.    Was  the  transition  from  the  previous  week (s)  to  this  week  of  the  Institute 
satisfactory? 

If  '*no,'^  please  comment  :  '  '       '  ' 


25.    Grade  this  past  week  of  the  Institute. 
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Recheck  your  5-digit  code  number  on 
page  1.     Thank  you. 

CIRCE 


SUMMAH  OPnilONKAIRE 


CIRCE 
Summer  1970 


5-DIGIT  CODE 

IRECTIONS:    Throughout  this  Institute  you  have  told  us  how  you  feel  about  a 
vonber  of  aspects  of  the  lnstruction;>l  program  of  this  Institute.    Now  we  are 
sklng  for  some  OVERALL  reactions ,    Check  one  response  category  for  each  item, 
nd  cononeat  if  you  wish  to  clarify  your  response. 

lease  use  the  same  S-dlgit  code  on  this  Oplnionnalre  that  >rou  have  been  uslnig 
hroughout  the  lest Itute. 

•    To  what  extent  do  you  feel  the  following  objectives  were  achieved: 

Very     Fairly     Mini-  Not 
Well       Itell       mally    At  All 


A«  An  understanding  of  the  problems  inhe rent 
In  the  management  of  an  educat ional  eval^ 
uat ion  project. 


□     □     □  □ 


b.  A  knowledge  of  methods  and  procedures  for 
,     evaluating  those  curriculum  changes  with 

which  you  as  an  Institute  participant  are  m  r— i  r-n 

concerned.  Q        Q         D  D 

c.  An  awareness  of  the  major  alternatives 
available  in  terms  of  educational  research 

methodologies.  □         Q         □  Q 

d.  An  understanding  of  the  relationship  of 
evaluation  to  the  problems  associated 
with  the  developm^ 

adoption  process  in  educational  innova- 

tion.;;;;  .';' _  Q^' 

e.  A  fdmiliarity  with  communications  tech* 
niques  applicable  :to  proper  iit^lementat 
of  the  dec  is  ion^making  process  at  various 

levels  of  the  educational  system^  CD        XH         d)  O 


You  probably  bad  soin<2  personal  objectives  which  were  not  the  same  as  those  of 
the  Institute*    If  so,  would  you  indicate  below  (briefly,  please)  what  they 
were  and  bow  well  they  were  achieved. 

Very     Fairly     Mini-  Not 
Well       Well       mally    At  All 

a*   .    •  ■ ' '  ' 


□     □     □  □ 


■  V  ;  ;         '     :  □     D     □  □ 


For  each  of  the  four  major  sections  of  this  Institute,  indicate  whether  time 
alloca^ied  was  <.oo  little,  about  right,  or  too  much: 


Too 
Little 

About 
Right 

Too 
Much 

a. 

Cook:    Orj*3  week  was— 

□ 

□ 

□ 

b. 

Walbess^r:    Two  weeks  was-- 

□ 

□ 

□ 

Stuff lebeam:    Two  weeks  was-- 

□ 

□ 

□ 

d. 

Asher:    One  week  was-- 

□ 

□ 

□ 
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4.    For  each  of  the  four  major  sections  of  the  Institute,  indicate  by  checking 
the  appropriate  box  the  extent  to  which  you  think  the  material  presented 
during  that  section  was  relevant  tc  your  own  activities  or  concerns: 


Very 
Relevant 

Somewhat 
Relevant 

Slightly 
,  Relevant 

Not 
Relevant 

a  • 

Section  One  (Cook) 

□ 

□ 

□ 

□ 

b. 

Section  Two  (Walbesser) 

□ 

□ 

□ 

□ 

c. 

Section  Three  (Stuff lebeam) 

□ 

□ 

□ 

□ 

Section  Four  (Ashe r) 

□ 

□ 

□ 

□ 

For  the  following  items,  please  mark  the  appropriate  box,  using  these  altematives 


SA  »  strongly  agree 

A  «  agree 

U  "  undecided  - 

D  »  disagree 
SD  -  strongly  disagree 


5.    The  content  of  this  Institute  has  direct  practical       SA       A       U       D  SD 
utility  in  my  institution.  Q    Q     D  C3^^^^^D 

Comment : 


6.    In  general,  the  approaches  presented  here  are  not         SA       A       U       D  SD 
feasible  in  my  institution.  Q     Q     [j    [j  Q 


Comment : 


ERLC 
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7/  The  benefit  I  derived  from  the  Institute  was  not  SA       A       U       D  SD 

worth  my  expenditure  of  time.  [*]     Q     D    Q  Q 


Comment: 


8.    This  Institute  compared  favorably/  overall,  with  SA     A     U     D    SD  Apply 

other  institutes  I  have  attended.  D   [Z]  D  Q   D  LJ 

.  Comment: 


9 •    The  diversity  in  content  and  approach  presented 
by  the  consultants  was: 

V' .„■■,''■■    ^  '  •  '      ■■..■.■/„':  sa\ 'A'  .■:u.  d^^sD' 

a.    Appropriate  to  the  objectives  of  the  Institute.     Q  [II    O  Q  t] 


Comment : 


b.    Satisfactory  to  me  personally.  SA       A       U       D  SD 

~  □  □  □  □ 


Comment 


c.    Too  great  for  me  to  be  able  to  assimilate  and         SA       A       U       D  SD 
integrate  things  as  well  as  I  would  have  liked.     Q     [_)     Q     [j  Q 

Comment:  '  ' 
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'f^.    If  this  Institute  were  to  be  offered  again,  I  would     SA      A      U      D  SD 
recommend  to  others  like  myself  chat  they  attend  it.    Q  Q  [j 


Conment : 


11.    Consultants  were  generally  not  available  for 

questions  and  discussion  outside  of  the  times  SA      A      U      D  SD 

scheduled  for  formal  presentations.  D    C3    D    D  Q 

ComnenL: 


12.    After  participating  in  the  Institute,  I  feel  more 
competent: 

a.  To  approach  and  conduct  evaluation  studies 

b.  To  design  and  carry  out  research  projects 


c.    To  develop  and  implement  sound  decision- 
making strategies. 


SA 

□ 

A 

□ 

u 

□ 

D 

□ 

SD 

□ 

SA 

A 

□ 

u 

□ 

D 
□ 

SD 

□ 

SA 
□ 

A 

O 

□ 

D 

□ 

SD 

□ 

13. 


The  follcwmg  cod.  U  usedlor      "clu.  he.dtgsr 
(I)  -  Project  manascasnt  rwdnls  r>r.d  techniques  (Cook) 

°  "0^ ^^^^^^^^^  on  p„.e.«U  si«U- 

°  -pt.n. 


a.  This  material  couid  have  considerable 

applicabiUty  to  iny  work  at  my  own 
institution. 

b.  I  really^ don't  see  how  this  could  apply  to 
the  problems  with  which  Ihave  to  work. 

c.  It's  likely  that  I  will  be  able  to  put 
this  to  good  use. 


e. 


f. 


in^;  ^  se^^i'ow' this  material  could  be  included 
in  another^Institute  in  the  absence  of  the 
consultant (s)  who  presented  it  here. 

This  material^ would  be^^^r^^^ 

to  other  Institutes  as  an  instructional 

ant(s)  who  presented  it  here  be  involved. 

I  wouidjecommend  that  my  colleagues  attend 
a^Wotkshop  or  Institute  which  had  this  topic 
as  its  central  focus.  ^ 


(1)  ,  (2)    (3)  (d) 
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How  would  you  describe  each  of  the  following  aspects  of  the  Institute? 
(Please  check  the  appropriate  response.) 


Excel* 

Un- 

Inad* 

Anil  0 

JCOQL 

Facilities 

Time  schedule 

Budget 

■r^—  

Format 

Materials 

1  ■■■J 

Equipment 

Staff  (Al  Elwell,  Mo  Oliver,  Ev  Barnes) 

Consultants 

Participant  Observers 

(Charles  Williamson,  Stuart  Pickard) 

Participants 

Food 

15.    Which  of  the  following  best  describes  your  overall  reaction  to  the  Institute? 

□  n 


Very        Favorable     Um'^i^  Unfavorable 
ravorable 

16.    A  six-week  time  allocation  for  an  InEtitute  like  this 


Very 
Unfavorable 

one  is: 


□ 


Hichtoo       A  little        About  A  little  Mu^Ttoo 


long  too  long        right  too  short  short 

^'  ^^J^'^^^^^^'^y -^^^^^\^or  the  staff,  consultants,  or  evaluators  of  this 


ERIC 


PARTICIPANT  OBSERVER'S  DAILY  LOG  NHUI 


I-  Paicilitators: 

events,  people,  decisions-- 

Arrangements  that  had  a  positive  effect  on  some  facet  of  the 
Institute: 


Hi  Inhibitors: 

events^  people,  decisions-- 

Arrangements  that  had  a  negative  effect  on  some  facet  of  the 
Institute- 


III.    Participants 'quotes: 

Quotable  quotes  that  reveal  participants'  views,  goals,  satisfactior^^ 
disappointments,  etc* 


IV,     Human  Relations: 

Especially  along  the  dimensions  of  age,  sex,  race,  and  lodging 
location. 
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Interviewee 


INSTITUTE  STAFF  SITE-VISIT 


INTERVIEW  FORM  Institute 


Date 

'^Greatest  strength: 


Interviewer 


*Greatest  weakness: 


^Problems  solved: 


*Problems  persisting: 


^Principal  intent  of  UNHI: 


*Freebee : 


EVALUATIOri  OF  THE  iNsf %CTI0NAl  EFFECTIVENESS  OF  THE 
1967  to  1970  SUIMEK  i^ESEARCH  TRAINING  INSTITUTES 
CONDUCTED  AT  THE  L^NtVEUSlTV  OP  ne'V  H/\MPSHIRE  , 


196?  -     Researching^  Cv^J-^ial  mucaHonal  Issues  in  Northevn 
New  tngland 

Researching  a  J  miiiating  Educational  Innovation 
tn  New  England 

The  Challenge  Of  As^es^ing  Curriculm  Chanaes  in 
iJew  England 

1970  -     Planning  for  sd^^aHonal  Decision-Making  in  the  ' 

(US 


1968 
1969 


INTRODUCTION 


Since  1967  the  Bureau  of  Educational  Rcs.e.arch  and  Testing  Services 
has  conducted  for  the  U.S.  Office  of  Hducation- a  sex.cs  of  four  (4)  sunder 
research  training  institutes.     In  1970,  .hile  considering  applying  for  the 
:  fourth  summer  institute  program,  the  Institute  Director  discussed  with  staff 
.embers  of  the  Research  Training  Branch,  -  USOI:.  the  possibility  of  funding  as 
part  of  the  fourth  summer  program,  a  participant  follow-up  study  of  all  four 
su^er  institutes.    The  general  purpose  for  the  follow-up  study  was  to  provide 
an- opportunity  to  gather  data  fro:n  past  participants  relative  to  the 
instructional  effectiveness  of  the  sun.ner  institute  progran.  Specifically, 
the  objectives  for  the  follow-up  study  were  stated  as  follows:     (l)  to  gain 
some  insight  fro.  past  participants  as  to  the  relative  strengths  and  weak- 
nesses of  the  summer  institutes  they  had  attended  and  (2)  to  determne  the 
value  of  the  instructional  programs  as  perceived  by  the  participants  during 
the  summer  institute  itself  and  as  viewed  from  their  present  Job  positions  in 
the  educational  profession. 

Therefore,  this  section  of  the  report  represents  two  different 
assessments  of  what  has  happened  to  the  participants  as  a  result  of  their 
involvement  in  the  summer  institute  programs. 
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The  geographical  representation  of  i.TiStit ate  participants  for  the  last 


GEOGRAPHICAL  REPRESENTATION  OP  INSTITUTE  PARTICiPAN" 


four  years  is  summarized  in  Tabic  A. 


Geographical  Reprc^janta Hon  of  Institute  'anticipants 

1067  to  1970 


Northern 

;]ew  Hajnpsltire     :Jev/  Eii.f^land       Nev/  England  National 

1967  1968  1969  1970 

************* -A'Ar*********:^*****  *******  **********  *  *  ************************** 

Connecticut  1  1 

Florida  2 

Georgia  2 

Hawaii  .  1 

Iowa  2 

Louisiana  2 

Maine                                       ^  6       •  2 

Massachusetts  2               .      3  :  3 

Michigan  ,  1  " 

Mississippi  2 

Montana  2 

New  Hampshire                         16  13  22  i 

North  Carolina  1, 

Rhode  Island  1 

Siapan  1 

South  Carolina  1 

Teniiessee  ^ 

Texas  2 

Virginia  .       1  . 

Vermont  ,       .                        10  .3  2 


30  25  29  29 


EKLC 


Statements  of  instructional  objectives  for  tlic  four  (4)  Summer 
Institute  Programs  are  presented  in  Appendix  C  of  this  report. 

The  first  and  most  extensive  part  of  this  section  of  the  report 
includes  a  presentation  of  the  results  based  upon  an  extensive  questionnaire  ' 
prepared  by  the  BERTS  staff.    The  questionnaires  were  sent  to  112  participants. 
Of  that  number,  we  received  a  response  froi.  67  participants,  for  a  return 
rate  of  60%.    One  of  the  major  problems  encountered  in  receiving  responses 
from  former  participants,  was  the  lack  of  accurate  forwarding  addresses. 
The  Bureau  attempted  an  extensive  check  on  the  mobility  of  participants  and 
only  as  a  result  of  this  effort  we  were  able  to  receive  the  number  of  responses 
indicated. 

TI)e  questionnaires  were  mailed  from  the  Bureau  of  Educational 
Research  and  Testing  Services  on  March  15,  1970  with  a  requested  return  deadline 
of  Hay  1,  1970.    The  return  rate  by  year  varied  considerably.    In  1967,  29 
participants  were  trained  and  we  received  14  responses  for  a  return  rate  of  48.3% 
In  1968,  25  participants  were  trained  and  we  received  9  responses  for  a  return 
rate  of  36^..    In  1969  29  participants  were  trained  and  we  received  21  responses 
•     for  a  return  r.'te  of  72.4%.    And  in  1970,  29  participants  were  trained  and  we 
received  23  responses  for  a  return  rate  of  79.3%. 

'  The  second  major  evaluation  effort  focused  on  an  on-site  ev'aluation, 

conducted  principally  by  a  participant-observer  in  the  1970  Institute  who  served 
as  a  field  evaluator.    The  follow-up  study. focused  primarily  on  site  visits  to  a 
g^-lected  number  of  1970  participants,  although  where  it 'was  possible,  the  BERTS 
staff  interviewed  participants  from  DTevinu<;  vnar^.  - 


IZVALUATION  OF  THE  IMSTRuC  FIONAL  nFFLCTIVliNBSS  OF  THE 
1967  to  1970  Shim'd  RESEARCH  TRAINING  INSTITUTES 
CONDUCTED  AT  TliT.  UNIVERSITY  OF  =:EW  HAMPSHIRE 


SECTION  I 

RESULTS  Of  PARTICIPANT  FOLLOW-UP  QUESTIONNAIRE 


FOLLOW-UP  EVALUATION  -QUESTION', jAIRE 

The  questionnaire  was  seventeen  (J 7)  questions  long  and  dealt 
with  various  aspects  of  the  institute  programs.     In  this  part  of  the  report, 
responses  to  e;,ch  question  are  followed  by  .a  brief  commentary. 

Question  1  --  How  many  pyogroms/pro.jeot  proposals  did  you  initiate  prior 
to  attending  your  institute? 

The  response  to  Question  1  across  the  four  years  would  seem  to 
indicate  .that  with  increasing  frequency  the  participants  had  in  fact  had  some 
experience  with  writing  proposals  previous  to  attending  the  institutes.  This, 
perhaps  represents  a  gro\dng  awareness  on  the  part  of  various  educational 
agencies,  of  a  need  for  compe  ency  in  this  particular  area. 


196? 

Proposals  (17) 
Participants  (14) 


1.2 


1968 

Proposals  (12) 
Participants       (9)  1.3 


1969 

Proposals  (  6) 

Participants  (21) 


1970 

Proposals  (48) 
Participants  (23) 


2.1 
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Question  2       Hou^  rnanu  vvogran/pvoject  pvoposaU  did  you  initiate/write  since 
attending  yoiu'  institute? 

The  response  pattern  to  this  question  seems  to  clearly  indicate 
that  the  number  of  proposals  written  by  the  participants  dramatically  increased 

subsequent  to  their  participation  in  the  institutes,     liven  in  so  short  a  period 

as  one  year  since  the  1970  summer  institute,  tliirty-six  (36)  proposals  have 
been  in  fact  crea^'ed. 

lOSl  1968 
Proposals  (17)  Proposals  (-53) 

Participants        (14)        2.1  Participants        (  9)  3.7 

1969  2970 
Proposals  (29)  Proposals  (36) 

Participants       (21)        1.4  Participants        (23)  1.6 

Question  3       Since  attending  the  institute  have  you  written/initiated  any  - 
of  the  following:    Proposals  (Federal,  State,  local) : 
evaluations:  research  studies? 

This  question  does  not  lend  itself  particularly  well  to  statistical 
treatment.     In  general,  it  is  safe  to  say  that  projects  written/submitted 
subsequent  to  their  involvement  in  the  institutes  were  oriented  more  towards 
state  and  local  activities.     It  is  fair  to  say  that  their  post-institute  efforts 
reflected  action  type  research  rather  than  formal  research  type  studies.  This 
would  be  in  line  with  the  general  professional  responsibilities  that  these 
ERIC    people  tended  to  carry  in  their  local  educational  or  state  agencies. 
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Question  4  --  %  coh  vole  Gxid/ov  function,  since  attending  the  -institute  lias 
changed  fi'orp.  to  ? 

Although  the  responses  to  this  question  do  not  necessarily  lend 
themselves  to  statistical  treatment,  th^  responses  are  imleed  worth  noting.  Thus, 
the  reader  is  referred  to  APPHNDlX  C,  for  a  listing  ef  participant  job  changes 
since  attending  the  institutes-.     Because  it  is  extremely  difficult  to  determine 
,a  direct  causal  relationship  between  Participating  in  the  institute  and  sub- 
sequent job  changes,  we  have  intentionally  avoided  a  detailed  analysis  of  job 
changes  (see  Question  n  \  fox  participants  perception  of  significant  job 
changes) .  .  ■ 

Question  5  -  VMt  is  the  present  status  of  the  program/project  proposal  you  ■ 
were  required  to  submit  as  part  of  your  Institute  experience? 

It  is  clear  from  the  data  below  that  from  a  third  (1/3)  to  one-half 
Cl/2)  of  the  proposals  developed  by  the  participants  were  pursued  to  completion. 
The  range  and  variety  of  funding  sources  which  are  reported  on  the  questionnaire 
are  too  numerous  to  present  here  but  let  us  offer  just  a  summary  of  relevant 
indexes. 

A  considerable  nuniber  of  these  projects  were  funded  as  part  of  local 
Title  I  efforts  botii  ir.  the  elementary  and  secondary  schools.    One' rather  large: 
Title  III  project  was  funded  in  excess  of  $45,000.    There  have  been  so  far,  two 
research  publications,  one  in  the  Science  Educator  and  one  in  the  Florida  Journal  ;, 
of  Educational  Research  which  were  direct  products  of  the  1970  institute. 
^Additionally,  one  doctoral  dissertation  being  completed  is  also  a  direct  outgrowth 
^I^Jf  involvement  in  the  1S70  Institute. 


It  would  soem  fair  to  generalize  from  the  responses  to  Question  5,  that 
the  participants  did  in  fact  pursue  the  researcli  and/or  evaluation  activities 
initiated  during  their  involvement  in  the  institute  prop^rajn.    The  majority  of 
the  proposals  have  in  fr.ct  been  implemented. 
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Active/ Implemented  (  3) 
Not  Impleinented  (  9) 


A c  t  i  V e / 1 np  1  eii) en t  e d  (  4 ) 
Not  Implenicnted  (  S) 


1969 


1970 


Active/ Implemented  (  ?} 
Not  Implemented  (12) 


Act  ive/Inp  lenient  od  (17] 
Not  Implemented  (  6) 


Question  6       Sinae  your  institute  cx'perirmce  hcje  you  been  in  contact  with 
(a)  institute  staff:  (b)  institute  oonsiiltants ^  (c)  other 
participants? 

The  response  data  to  question  6  suggest  two  rather  clear  patterns. 
One,  the  participants  judged  that  tlio  institute  staff  and  to  a  lesser  degree'  the 
consultants  were  indeed  available  to  them  in  post-institute  contacts.    Two,  the 
participants  also  indicate  that  post-institute  interactions  with  other 
participants  were  greater  than  botli  the  institute  staff  and  consultants.  This 
is  in  fact  what  one  would  generally  tend  to  expect.     (Undoubtedly  the  participants 
in  tlie  1967,  1968  institutes  are  not  so  limited  by  geographical  proximity.) 

The  second  part  to  Question  6^  attempted  to  identify  the  specific 
context  in  which  post-institute  contacts  were  made.    The  responses  were 
varied.    A  seriec  g:"*  rcprc:ont'itive  comments  follows: 
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Institute  Staff: 


"Help  with  projects" 
"Workshop  at  UNH" 
■'Research  proposal  ' 

"Feasibility  of  computerizing  spelling  program" 


Institute  Consultants 


'Writing  of  articles" 
"Assisting  me  on  my  dissertation' 


Other  Participants : 


hired  him  as  my  assistant  principal" 
"I  traveled  to  Rome  with  one" 
"In  formulating  proposals'" 
"Exchanged  notes''  " 

"Contacted  7  others  to  see  if  they  had  results 

on  their  proposals'' 
"Research  information  and  data  on  specific 

projects'' 

^|Conducted  evaluation  workshops  with  participants'' 
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1968 

1969 

1970 

TOTAL 

Institute  Staff  ' 

YES 

6 

(42.9%) 

8 

(88.9%) 

10 

(47.6%) 

13 

(56.5%) 

37  (52.2%) 

NO 

8 

(57.1%) 

1 

(11.1%) 

11 

(52.4%) 

10 

(43.5%) 

30  (40.8%) 

Institute  Consultants 

YES 

4 

(28.6%) 

6 

(66.7%) 

7 

(33.3%) 

16 

(69.6%) 

33  (49.3%) 

NO 

10 

(71.4%) 

3 

(53.3%) 

14 

(66.7%) 

7 

(30.4%) 

34  (50.7%) 

Other  Participants 

YES 

8 

(57.1%) 

7 

(77.8%) 

14 

(66.7%) 

22 

(95.7%) 

51  (76.1%) 

NO 

6 

(42.9%) 

2 

(22.2%) 

7 

(33.3%) 

1 

(  4.3%) 

16  (23.9%) 

Question  7  -  Since  attending  the  institute  have  you  been  called  upontooonduct 
in-sevvice  training  progratns,  workshops,  ov  simply  give  presenta- 
tions in  areas  the  institute  concentrated  on?' 
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The  response  pattern  to  Question  7  would  suggest  that  approximately 
46%  of  the  participants  have  rather  consistently  been  asked  to'  provide  additional 
training  of  an'  in-service  nature.    Note  that  the  number  of  yes  responses  has 
steadily  increased  over  the  years,  from  a  low  of  21%  in  1967  to  a  high  of  60% 
in  1970. 

This  does  seem  to  reflect  a  trend  on  the  part  of  state  and  local 
educational  agencies  to  attempt  to  upgrade  the  evaluation  capabilities  of  their 
local  staffs.    Another  related  interpretation  could  be  that  the  steady  increases 
reflect  the  fact  that  in  all  of  the  four  years  the  trainee  was  required  to  receive 
a  recommendation  from  his  or  her  immediate  supervisor  before  being  considered 
for  selection  as  a  participant.    As  such,  levels  of  expectation  in  the  part  of 
superiors,  relative  to  participant  performance  tend  to  increase. 


YES 

m  . 

1967 

3 

(21.4%) 

11 

(78.6%) 

1968 

3 

(33.3%) 

5 

(55.6%) 

1 

(11.1%) 

1969 

11 

(52.4%)- 

9 

(42.9%) 

1 

(  4.8%) 

1970 

14 

(60.9%) 

9 

(39.1%) 

TOTAL  . 

31 

(46.3%) 

34 

(S0.?%) 

2 

(  3,0%) 

The  second  part  to  Question  7,  attempted  to  identify  the  specific 
number,  content  and  audience  of  such  activities.    Sample  responses  are  as 
follows : 

"One  presentation  to  the  Timberlane  School  Board." 

'=20  School  Guidance  people  -  "Colloquium  on  Innovation  and 
Direction  in  Guidance  in  N.H." 
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=  'At  State  Department  of  Education  for  75  members  of  staff 
concerning  CIPP  evaluation  model." 

"6  sessions  behavioral  objectives  for  10  Department  personnel." 

"One  presentation  to  Department  staff  members  in  PERT  and  methods 
of  evaluation.'' 

Three  in  context  evaluation  --  State  Department  employees, 

Chief  State  School  Officer  and  assistants,  teachers  and 
administrators."  '  ' 

"Two  workshops  for  Title  I,  HI  and  IV  projects  for  Directors 
and  state  agency  staff.''.  ' 

Question  8  -  Have  you  taken  or  been  involved  in  courses,  seminars,  workshops, 
etc.,  since  your  institute,  related  to  educational  research 
and/ or  educational  evaluation? 

The  response  pattern  suggests  that  generally,  from  a  third  to  a  half 
of  the  participants  sought  further  training  in  either  seminars  or  workshops  ■ 
dealing  with  evaluation  and  research. 


YES 

N0_ 

1967 

4 

(28.6%) 

10 

(71.4%) 

1968 

4 

(44.4%) 

5 

(55.6%) 

1969 

7 

(53.3%) 

14 

(66.7%) 

1970 

12 

(52.2%) 

11 

(47.8%) 

TOTAL 

27 

(40.3%) 

40 

(59.  7%) 

The  second  part  of  Question  8,  attempted  to  identify  specific 
dates,  title  of  seminars  and  place.    A  series  of  sample  comments  follow; 
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^'Attended  week  long  workshop  in  Wiii-^.,  to  study  behavioral 
objectives  in  reading  --1969'^ 

••DISP  Hxecutive  tcara  retreat       5  days  on  accountability 
1970.;' 

'^Re;earch  Practicum,  Systems  Evaluation  in  Educational 
Decision-Making       L]nivGr::;ity  of  Iowa  1970'' 

■"Waco,  Texas       November  1970       Professional  Growth  Program 
Waco,  Texas  --  March  1971  EducationaJ  Research  Project'* 

^'Doctoral  Program       University  of  Michigan  1967-19'69" 

"Yes       as  components  of  conferences .  ^' 

"Multi-variate  statistics       Nova  University  1971'' 

''6  hours  in  research  as  Ph.D.  candidate  --  University  of 
iMiami       September  1971" 

Through  1972       doctoral  studies  in  evaluation,  research 
and.  educational  development,^' 

■'University  of  North  Carolina  ^-  Fall  course  in  Educational 
Research  1971'  = 


Question  9  —  7v7?at  use  have  you  made  of  the  instructional  materials  presented 
to  you  at  your  institute? 


A  representative  list  of  responses  follows: 

1.  They  tended  to  be  used  as  references  and  particularly 
to  help  others  learn  some  of  the  mysteries  of  writing 
proposals, 

2.  They  were  used  specifically  as  personal  referencesdn 
more  advanced  classes  taken  later  in  graduate  work, 

3.  Use  of  the  materials  in  conducting  teacher  workshops, 
specific  mention  was  made  on  the  use  of  PERT  and  the 
establishing  of  behavioral  objectives  for  curriculum 
revisions  that  were  being  planned. 
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4.  The  CIPP  e.v.aliiation  model  has  been  commonly  discussed 
m  a  variety  (Of  workshops  and  teacher  evaluation 
seminars  and  ipresentations  made  to  a  number  of  local 
administrative  groups  interested  in  up-grading  their 
evaluation  skills. 

5.  And,  of  course,  for  some,  the  materials  have  been  placed 
in  "perraanent  files." 


Question  10  -  Due  to  the  extensive  analysis  conducted,  Question  10  will 
be  discussed  later  on  in  the  report. 

Question  11  -  Do  you  feel  that  your  job  function  or  role  has  changed  ae  a 
Result  of  your  participating  in  the  Institute? 

The  respondents  for  l967  and  1968  seem  to  fall  into -two  quite 
different  categories.    Inl967  (14%)  and  1968  (33%)  saw  it  as  very  significant, 
while  for  the  same  years,  50%  and  44%  saw  it  as  somewhat  significant.  During 
the  years  1969  and  1970.  65%  and  60%  respectively,  of  the  respondents  saw  it 
as  significant  or  very  significant. 

Tils  response  pattern  begins  to  suggest  that  with  increasing  emphasis 

people  who  participate  in  summer  isese^ch  institutes  view  their  activities  as 

more  iinportsmtrithan  f hey  have  been  viewed  in  the  sever^^^  : 

^.fy   ^     ^.  Somewhat 
Stgmftcant     Stgnificant      Significant        Not  At  All      Don't  Know 

1967              ^(14.3%)              ...  7  (30.0,3  4  (28.6%)  1  (7.1%) 

''''              ''''■''^  4(44.4%)  1(11.1%)  i(n.l%) 

^^^^              "^''-^'^  10  (47.6%)  6  (28.0%)  1  (  4.8%)  _  ; 

4(17,4%)  10(43.5%)  5.(21.7%)  3(13.0%)  1(4.4%) 


^'^'otal  13(19.4%)        20  (29.9%)        22  (32,8%)  9(13.4%) 
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Question  12      Md  you  oomit  yPU^s^lf  to  a  nmbef  of-  such  Institutes  (6- 
week)  to  reoeive  af^  ad^an^ed  degree  suoh  as  a  Master 's  in  Edu- 
cational Research  i!^'^d/oy  Education? 

Approximately  60-65%  of  %  respondents  indicated  that  they  would 
in  fact  coramit  themselves  to  the  J-nsutute  method  of  .earning  an  advanced  degree. 
Very  few  of  the  participants  gave  ^  definite  no  and  a  significantly  large  number 
(approximately  40%),  particularly      the  years  -67,  '69  and  '70  said  maybe. 


Yes 

Maybe 

1967 

6  (42.^'^) 

2  (14.2%),^ 

6  (42.9%) 

1968 

6  (66.^^°) 

2  (22.2%) 

1  (11.1%) 

1969 

11  (52.4^^) 

2  (9.5%) 

8  (38.1%) 

1970 

10  (43./«) 

^  (26.1%) 

7  (30.4%) 

Total 

33  (49jh 

22  (27,9%) 

22  (22.8%) 

Question  13      Participants  to;-.Ing^itwteS  of  :tHB  type  should  be  grouped 

by  experience,  ^ob^ij^Hon  and  expertise  on  a  more  homogeneous, 
heterogeneous  bas-vsf^r  no  <^i^on7 

For  the  years  1967  and  i^^s  the  participants  seemed  to  be  fairly 
.eVenly  divided  between  homogeneous?  M  heterogeneous  grouping.    In  1969  there 
is  a  considerable  switch  in  opinio^  Wjth<approximately  43%  indicating  they 
fewortd  heterogeneous  and  only  10%  favoj-ing  homogeneous  grouping.    In  the 
1970  institute  it  switched  back  to        of  the  participants  indicating  they 
supported  homogeneously  group  part^^ipajits. 
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Homogeneous 

Heterogemous 

No  Opinion 

.1967 

5  (35.7%): 

6  (42.9%) 

3  (21.4%) 

1968 

4  (44.4%) 

3  (33.3%) 

2  (22.2%) 

1969 

2  (  9.5%) 

9  r42  95i1 

1970 

15  (65.2%) 

6  (26.1%) 

2  (  8.7%) 

Total 

26  (38.8%) 

24  (35.8%) 

17  (25.4%) 

Qae.stion  14  -  Z?o  you  fee^  t?ie  use  of  "6%  consuZtants  /rom  Z^^e^  out- 

standing  universities  at  tliese  institutes  is  of  extusme  value^ 

or  sometMng  that  aouldMB  handl 

comparably  trMined  personnel,  of  signv^^^ 
no  opinion? 

For  1967  and  1968  the  majority  of  the  partixlpants  respoading  felr 
tifet  it  could  be  handled  as  effectively  by  less  well  known  people.    Beginmng  in 
m69  there  seemed  to  be  a  rather  marked  switch  in  opinion  jsoirthat  by  197Bis. 
E^jority  of  the  respondents  :in  fact  feitithat  the  use  ofrrimportant  consulitants 
firom  large,  outstanding  universities  wasK  of  extreme  ualue. 

MB^Hed  by.  No 
Extreme  '^Sess  Significant  No 

'    l^iue  Weimmom  Value  Opinion 

1967  ■  5  (S17%)    ;     '9  P4.3%)  — 

1968  2  GZ2.2%)  S:t055.6%)         2  (22-2%) 

1969  8  (M.1%)         l'0  f47.6%)         2  (  9S5%)  1(4.8%) 

1970  ■  13(56.5%)  7(30.4%)  1(4.4%)  2(8.7%) 
Total                28  (41.8%)        32  (.46.3%)         S  (  7.5%)         3  (4.5%) 
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Question  15  —  What  are  some  of  the  changes  you  would  make  assuming  you 
Were  conducting  th^  Institute? 

This  question,  along  with  Questions  16  and  17,  do  not  lend  them- 
selves to  quantification.    Therefore,  we  shall  indicate  only  a  few  representative 
responses: 

"More  structure'* 

"Greater  emphasis  on  statistiHcs*^ 

"Lengthen  tSe  institute'' 

"Make  computer  programming  op^tEorsal" 

"Some  type  :ef  fol  low-up  actx^tsasaes  to  encourage 
further  interest" 

"Wore  .emphai^is  on  how  to  write  a  proposal 

"More  tdmeiSfbr  individual  resesao^ 
reading" 

.  ■  ...  .  ■ 

"More  empjkasis  on  '  problem-sola^iIi^g  *  rather  than 
specialisation" 

"More  participant  consultant  iTrtexactian" 

"More  careful  selection  of  iparMcipants'' 

"Try  to  involve  the  agencres^:^evparticipants 
represent" 

"More  homogeneous  grouping" 

Question  16      What  do  you  believe  the  greatest  strengths  of  such  an 
Institute  are? 

The  general  consensus  of  the  responses  to  this  item  center  around 
two  areas;  interaction  and  professional  growth.    Numerous  participants  stated 
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that  they  felt  the  "exchange  of  information"  with  fellow  participants,  BERTS 
staff  ai^d  the  consultants  was  of  extreme  value.    The  ability  to  exchange  ideas, 
both  :f o.n,a.H:y  and  informally,  with  fellow  professionals  and  to  solicit  assistance 
fro.  others  who  may  have  been  more  experienced  seemed  to  be  overwhelming  strength. 
o£  the  Sastitutes.    Second  to  this  was  regarded  as  professional  growth, 
■'<¥'  datSBB  roneself','.  the  exposure  to  new  materials  and  the  use  of  real  life 
simua:at2«ms  for  problem  solving.  . 

Question  17  -.^^  .f-Z/^t  ^        Ulieve  the  greatest  maKnesses  of  such  instates 
are? 

Once  again:,  typical  responses  seemed  to  be: 

"!ibo  much  material  in  too  short  a  time" 

■■'ITbo  much  emphasis  on  computer  program  wri ting'' 
^'tiJot  enough  use  of  planned  "team  learning" 
"Bogging  down  on  course  lectures" 
'Ifeck  of  follow-up" 

"The  variance  of  personal  backgrounds" 
"Direction" 

'p"JoJec1  inlepJh'-'"''"  '"^ 

The  statements  recurring  most  frequently  seem  to  be  the  ones  stating 
that  the  institute  attempted  to  accomplish  too  much  in  too  short  a  period  of 
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time  thus  giviiiEg.fa3ily  superficial  attention  to  the-  instructional  areas  and 
secondly  the  J  a-Gfea:£:a     type  of  follow-up  such  as  seminars  or  workshops  to 
help  partici5aBtl2Si!ine.et  real  life  problems  they  have  trying  to  implement  institute 
materials-  or  xisnGigjts  back  in  the  field. 


Question  10  -~Mms:e  rate  the  following  Instruotional  areas,  as  presented 
^■^0^  Institute,  ACCORDING  TO  THEIR  IMPORTANCE:    (a)  Wfiile 
.Mi:},smBng  the  Institute  and  (b)  to  your  present  job  position. 

'Tfeisasponse  analysis  which  follows  is  rather  extensive.    Part  I 
is  based  upaiEsaj  apalys is  of  the  responses,  for  all  the  participants  (total 
Group),  to3tfe««toeen  (13)  sub-parts  to  Question  10.    The  response  rates  are 
reported  as  Sf&Ms:: 

(a) )  SESScipants  were  asked  to  rate  on  a  four  point  scale  -- 

asEiranely  important,  important,  minimally  important  and 
a^|-2important  at  all  --  the  value  of  the  instructional 
ai^wiE,  as  perceived  by  the  participants  [During  the 
ifeissiiia:ute]  itself  and  as  viewed  from  their  [present  job 
jgBx^irtaLons]  in  the  educational  profession. 

(b)  Response  patterns  are  reported  in  percentages. 

(c)  A  bEief  commentary  on  the  response  patterns  is  made 
:lox:.E3:ch  sub-part  (lOA-lOM)  and  associated  data  tables 
ClOA^r^iDM)  immediately  follows. 


?ART  I 

TOTAL  GROUP  RESPONSES  BY  PERCENTAGES  — 
VALUE  OF  INSTRUCTIONAL  AREAS  AS  PERCEIVED  DURING  THE  INSTITUTE 
AND  FROM  THEIR  PRESENT  JOB  POSITIONS 


Question  lOA      Developing  and  writing  behavioral  objectives.  [During 
the  institute],  78%  of  the  respondents  viewed  this 
instructional  area  as  important  to  extremely  important. 
From  their  [present  positions] ,  72%  of-  the  respondents 
■  still  viewed  the  instructional  area  as  important  to 
extremely  important. 


Question  10  B      The  oonstruation  and  identification  of  learning  hierarchies. 

[During  the  institute],  in  excess  of  60%  of  the  partici- 
pants viewed  this  instructional  area  as  important  to 
extremely  important.    From  their  [present  positions], 
approximately  40%  felt  the  instructional  area  was  im- 
portant to  extremely  important.    The  response  pattern  would 
seem  to  indicate  a  fair  degree  of  ambivalence  towards  the 
importance  of  this  instructional  area. 


TABLE  10  A 


During  the  Institute 


Extremely 

Miflimaliy 

Not 

No 

1967 

Important 

ImiDortant 

Important 

Important 

Opinion 

2-14'.  3% 

8-.S7.2% 

1-7.1% 

3-21.4% 

1968 

1-11.1% 

6-66. 7'i 

1-11.1%  , 

1-11.1% 

1969 

8-38.0% 

1 1  .i;2  4% 

1    A  ^9' 
i  -    .  0  -s 

1-4 . 0^ 

1970 

5-21.7% 

12-S2  2% 

♦  /    i  0  .  u  0 

e^-^  0  .  /  1) 

I -4 . 

TOTAL 

16-23  9% 

6-9.0% 

y  T*  A  C  OTI  ^ 

I  Ubl Lion 

Extremely 

Miniinallv 

Not 

No 

1967 

Important 

V.Tiportant 

Important 

Opinion 

3-21.4% 

V?    O.J  •  0  i) 

i  -  / .  1  -ti 

4-28,6% 

1968 

3-33.3% 

2-22.3% 

1-11.1% 

1969 

8-38.0% 

9-42.9% 

1-4.8% 

2-9.5% 

1-4.8% 

1970 

8-34.9% 

10-43.5%  • 

1-4.-3% 

1-4.3% 

TOTAL 

22-3^ . 8% 

27-40. 

7-10.4% 

4-6.0% 

7-10.4% 

TABLE 

10  B 

During  the  Institute 

Extremely 

Minimally 

Not 

No 

1967 

Important 

Important 

Important 

Important 

Opinion 

9-64.  :n 

2-14.3% 

3-21.4% 

1968 

1-11.1% 

4-44.4% 

1-11.1% 

1-11.1% 

2-22.3% 

1969 

3-14.3% 

11-52.4% 

5-23.8% 

1-4.8% 

1-4.7% 

1970 

3-13.0% 

11-^47.8%' 

8-34.9% 

1-4.3% 

TOTAL 

7-10.4% 

35-52.2% 

16-23.9% 

3-4.5% 

6-9.0% 

Fiesent  Po;;ition 


'  •■  Extremely  Minimally  Not  No, 

Important  Important  •  Important  Important  Opinion 

1967  1-7.1%  6-42.9%  1-7.1%        '      2-14.3%  4-28.6% 

1968  1-11.1%  1-11.1%  3-33.4%  2-22.2%  2-22.2% 

1969  -  8-38.1%  9-42.9%  3-14.3%  1-4.7% 

1970  3-13.0%  '6-26.1%  10-43.5%  3-13.0%  1-4.4% 
TOTAL  5-7.5%  21-31.5%  23-34.3%  10-14.9%  8-12.0% 
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Question  IOC  --  Understanding  the  various  administrative  approaches  to 

■      program,  project  management.     [Durifig  the  institute],  80% 
of  the  participants  viewfid  this  instructional  area  as 
important  to  extremely  important.     It  is  interesting  to 
note  that  in  1968,  only  55%  of  the  participants  viewed 
the  instruction  as  important  to  extremely  important.  On 
the  other  hand,  95%  of  the  participants  in  the  .1970 
institute  sav^^  it  as  important  to  extremely  important. 
From  their  [present  position],  70%  of  the  participants 
still  viewed  this  instructional  area  as  important  to 
extremely  important. 


Question  lOD  --  The  skills  acquired  in  utilizing  the  program  evaluation 
review  technique  (PERT)..    [During  the  institute], 
approximately  60%  of  the  participants  indicated  the 
instructional  area  was  important  to  extremely  important. 
The  remainder  viewed  it  as  minimally  important  and  3% 
viewed  it  as  not  important.    From  their  [present  position], 
approximately  48%  of  the  participants  viewed  the  use  of 
PERT  as  important  to  extremely  important.  Approximately 
40%  saw  it  as  being  minimally  important  to  important. 
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TABLE  10  C 


DuTins  t:he  Institute 


1967' 

1968 

1969 

1970 

TOTAL 


1967 
1968 
1969 
1970 
TOTAL 


Extremely 
important 


2-14.3% 
5-33.3% 
5-23. 3*-i 
7-30.4% 
17-25.4% 


Extremely 
Important 


4-28,6% 

2-22.2% 

4-19.0% 

10-43.5% 

20-29.9% 


Iinportant. 

 -r  


8-57. 1  . 
2-22.25- 
12-5; . 1% 

is-(ys.2% 

37-55.2% 


Minimally 
Tniportsnt_ 
1-14.::;%" 
2-22 , 
.••'-14.  3-^ 
l-A.4% 
1.9% 


Pre=;en-:  Position 


Inrpoi-i:  ant 


4-28.6% 
2-22.2% 
11-52.4% 
.M  -47,S% 
28-41.8% 


Miniiv;.n  I  ly 
I_npo-ftant 
"2-^4.3"^  ~ 
l-n.1% 
4-19.0% 
2  - S . 7% 
9-13.4% 


Not 
important 


Iinportant 


.5-7.1% 

2-22.2% 

1-4.8% 

4-6.0% 


Opinion 
2-14.3% 
2-22.5% 
1-4.8% 

5-7.S% 


No 
Opinion 


3-21.4% 
2-22.3% 
1-4.8% 

6-8.9% 


1967 
1968 
1969 
1970  - 
TOTAL 


1967 
1968 
1969 
1970 
TOTAL 


TABLE  10  D 


Extremely 
Important 

1-  7.1% 
3-33.3% 

2-  9.5% 
8-34.8% 
14-20  9% 


Extremely 
Important 


1-  7.1% 

2-  22.2% 
4-19.0% 
4-17.4% 
11-16.4% 


During  the  Institute 


_niTgortant 
7-50,0% 
3-33.3% 
11-52.4% 
9-39.1% 
30-44.8% 


Minimally 
Important 


1-  7.1% 

2-  22.3% 
5-23.8% 
5-21.7% 
13-19.4% 


Present  Position 


Important 
'3-21.4% 
2-22.2% 
4-19.0-5 
12-52.2% 
21-31.  3'.i 


Miniin.-ally 
J^mgortant 

3-  2l74% 

4-  44.4% 
8-38. 1^- 
7 -.3  0.4% 


Not 
Important 


1-4.8% 

1-  4.4% 

2-  3.0% 


Not 
Important 


1-7.1% 

3-  14.. 3% 

4-  6.0% 


No 
Opinion 


5-35.8% 

1-  11.1% 

2-  9.5% 

8-11.9% 


No 
Opinion 


6-43.0% 

1-  11.2% 

2-  9.6% 

9-13.4% 
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(^\c^<in  lOE^-  The  CIPP  Model  of  Evaluation  (Context,  Input,  Vvocass  and 

■      '  Product).     ■  .  ;  . 

[During  the  institute],  the  response  pattern  indicates 
thu  the  first  two  years,  participants  were  not  real'fy 
exposed  to  CIPP  since  it  was  just  in  its  forinative  stages. 
For  most  of  the  responses,  their  knowledge  relative  to 
CIPP  was  acquired  through  contacts  with  other  participants, 
the  BERTS  staff,  and  attending  conferences.    In  generdl,    '  • 
approximately  60-5  marked  it  as  iniportant  to  minimally 
important.    The  picture  seems  to  change  very  markedly  by  1970 
however,  when  ^Vzost  90%  of  the  participants  viewed  the 
instruction  as  important  to  extremely  important.  From 
their  [present  position],  the  response  pattern  is  similar. 
The  participants  in  earlier  institutes  had  not  been  exposed 
to  CIPP.    The  entire  1970  group  of  respondents  viewed  CIPP 
as  important  to  extremely,  important. 

(2"^g^ion  lOF      The  skillj  and  ppaotice  acqutpad  by  writing  computer  programs. 

[During  the  institute],  the  response  pattern  is  quite  interest 
ing  since  it  seems  to  indicate  that  over  the  years  the  ability 
to  write  computer  programs  has  become  less  valuable.  This 
perhaps  reflects  th<^  increasing  number  of  computer  programmers 
that  are  available  to  professional  educators  thereby-  lessening 
the  need  for  the  personal  skill  at  writing  computer' programs . 
The  participant  responses  range  from  a  high  of  56%  in  1967  to 
a  low  of  21Vin  1970  who  viewed  the  instruction  as  important 
to  extremely  important.    From  their  [present  position] ,  a 
similar  response  pattern  is  noted  but  even  more  pronounced. 
The  participants  seem  to  be  saying  tha'c  busy  school  people 

c^o  not  have  the  time  to  ^n  fact  write  their  own  computer 
programs. 

This  phenomenon  had  been  observed  by  the  institute  staff 

over  the  years  and  appropriate  instructional  modifications 
have  taken  place.    Fur  instance,  ir/ the  1970  program, 
computer  programming  was  made  optional  to  the  participants. 
Seemingly,  the  emerging  shift  is  towards  the  use  of  terminal 
devices  which  school  administrators  can  use  with  much  less 
comprehensive  knowledge  of  the  abilities  of  writing  computer 
programs. 


TASLE  10  ^ 
Durins  the  Institute 


Extremely 

Mininially 

Not 

No 

1967 

Important 

Important 

Importar*t 

Important 

Opinion 

.--35.  n. 

^-21.4% 

6-42.9% 

1968 

1-11.1% 

5-55.6% 

2-22. 2^' 

1-1M% 

^  -  y .  h  5 

.  2>9,5% 

1970 

17   r/.i  l^Ia 

JL  -    .  .  CI 

lUlAL 

12-1/ .9% 

10-14. 9'i 

1-1.S% 

9-15.5% 

PrGf:e:it 

Positio?. 

Extremely 

Not 

No 

1967 

Important 

,  li^pcrtanr: 

T -.rip  or  t  ant 

Opinion 

i-7.1% 

6-42.9%- 

1.968 

.  1-11.1% 

2-22.2% 

3-33. 4 V 

2-22.2% 

1-11.1% 

1969 

5-23.2% 

3-14.31 

.*.)-*  14  ,  ^>°6 

3-14.3% 

1970 

11-47.8^, 

TOTAL 

17-25.4% 

23- 34. 3% 

11-16,4% 

6-S . 0% 

10-14.9% 

During  the  Institute 


Extremely 


Not 


liiiportar 


mportant 


No 


1967 

2-14.3% 

6-42.9% 

4-28.6% 

2-14.2% 

1968 

4-44.4% 

1-11.1% 

2-22.2% 

2-22.3% 

1969 

2-9.5% 

10-47.6% 

7-33.3% 

2-9.6% 

1970 

1-4.3% 

4-17.4% 

3-13,0% 

3-13.0% 

.  12-52.3% 

TOTAL 

9-13.4% 

21-31.3% 

16-23.0% 

7-10.,4% 

14-21.0% 

Present 

Position 

Extremsly 

Minimally 

Not 

No 

Important 

Important 

Important 

Ijuportant 

Opinion 

1967 

1-7.1% 

.  3-21.4% 

4-28.6% 

5-35,7% 

1-7.2% 

1968 

^-22.2% 

3-33.5% 

2t22,2% 

2-22.3% 

1969 

i-4.8% 

6-28.6% 

6-28.6% 

6-28.6% 

2-9.4% 

1970 

2-8.7% 

■  2-S.7% 

5--21.7% 

2-8.7% 

12-52.2% 

TOTAL 

6-9.0% 

14-20,9% 

17-25.4% 

15-22.4% 

15-22.3% 
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Question  lOG      The  skills  and  practices  acquired  in  operating  computers 
and  computer  supported  hardware.    [During  the  institute] , 
in  excess  of  60%  of  the  institute  participants  saw  this 
area  as  minimally  important  to  important.    From  their 
[present  position],  approximtely  50%  of  the  partici- 
pants judge  it  to  be  minijnally  important  to  important 
to  have  these  skills  in  their  present  position.  Again, 
perhaps  reflecting  the  same  reasoning  as  discussed  in 
Question  lOF. 

Question  lOH      The  statistical  skills  offered  as  an  introductory  base 
or  general  review  of  those  needed  in  research.  [During 
the  institute] 5  for  the  first  two  years  some  70-80%  of 
the  participants  saw  this  as  an  important  to  extremely 
important  skill.    In  1969,  75%  of  the  participants  saw 
it  as  minimally  important  to  important  and  in  1970,  only 
approximately  40%  saw  it  as  important,  minimally  important, 
or  extremely  important.    In  their  [present  position],  the 
response  pattern  is  quite  similar.    Again  this  skill  is 
J         perceived  by  the  participants  over  the  four  year  period  as 
becoming  less  important.    This  would  seem  to  be  a  similar 
finding  to  the  response  noted  for  lOF  on  computer  skills 
where  the  participants  are  seeing  perhaps  the  need  to  have 
'  consultants  or  highly  trained  people  capable  and  available 

to  them  but  not  to  acquire. and  master  the  skills  themselves 


TABLE  10  G 


During  the  Institute 


1967 
1968 
1969 
1970 
TOTAL 


Extremely 
Important 


1-  7.1% 

2-  22.2% 


3-4.5% 


[inportant 


6-  42.9% 
3-33.3% 

7-  33.3% 
2-8.7% 
18-26.9% 


.Minimally 
Important 


5-35.7% 
5-33.5% 
10-47.6% 
5-21.7% 
23-34.3% 


Not 
Important 


2% 


1-7.  l?c 

1-  1 

2-  9. S% 
4-17.4% 

3-  12.0% 


No 
Opinion 


1-7.1% 


2-9.6% 

12-52.2% 

15-22.3% 


Pi-esent  Position 


1967 
1968 
1969 
1970 
TOTAL 


Extremely 
Important 


2-22.2% 


2-3.1 


Imjxjrtan-; 


4-28.6% 
4-44.4% 
6-28.6% 
2-8.7% 
16-23.9% 


Minimally 
Important 


4-28.6% 
2-22.2% 
7-33.3% 
4-17.4% 
17-25.3% 


Not 
Important 


5-35.7% 
i-11.2% 
5-28.6% 
4-17.4% 
16-23.9^ 


No 

Opinion 


1-  7.1% 

2-  9.5% 
13-56.5% 
16-25.9% 


*iNr**ilr****************f******  ********* 

TABLE  10  H_ 
During  the  Institute 


1967 
1968 
1969 
1970 
TOTAL 


Extremely 
Important 


S-35.7% 
4-44.4% 
2-9.5% 
2-8.7% 
13-19.4% 


Important 


8-57.1% 
3-33.3% 
ll-.52.4% 
6-26.1% 
28-41.8% 


Minimally 
Important 


1-  7.2% 

2-  22.3% 
5" 23. 8% 
2-8.7% 
iO-14,9% 


Not 
Important _ 


1-4.8% 

4-  17.4% 

5-  7.5% 


No 

Qioinion 


2-9.5% 

9-39.1% 

11-16.4% 


Prss  ent  Poz it  ion 


1967 
1968 
1969 
1970 
TOTAL 


Extremely 
Important 


3-21.4% 
3-33.3% 

3-  14.3% 

4-  17 -.4% 
13-19.4% 


Important 


Minimally 
Important 


Not 
Important 


9-64.3% 
2-22.2% 
9-42.9% 
6-26.1% 
26-38.8% 


4-44. S% 
6-28.6% 
2-8,7% 
12-17.9% 


1-7.1% 


1-4.8% 
1-4.3% 
3-4,5% 


No 
Opinipn 


1-  7.2% 

2-  9.4% 
10-43.5% 
13-19.4% 
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Question  101  —  Knowledge  gained  from  interaction  with  fellow  pai^ticipants . 

[During  the  institute] ,  from  70-80^3  of  the  respondents 
indicated  that  this  was  important  to  extremely  important. 
This  would  argue  cogently  for  future  institutes  providing 
adequate  amounts  of  time  for  participant  interaction 
(e.g.,  team  learning).    From  their  [present  position],  the 
response  pattern  seems  to  be  similar.    The  general  ratings 
being  from  60-70%  viewing  it  as  importani  to  extremely 
important. 


Question  lOJ  r-  Knowledge  gained  from  interaction  with  BERTS  staff, 
(exclusive  of  outside  consultants) 
[During  the  institute],  approximately  85-5  of  the 
participants  rated  this  as  important  to  extremely 
important.    From  their  [present  position] ,  approximately 
60%  of  the  participants  viewed  interaction  with  the 
BERTS  staff  as  important  to  extremely  important. 


ERIC 


TABLE  10  I 

During  the  Instituto 

Extremely  ■Minimlly  Not  'No 

In^pc'ytant         Importa-)-:' .   zey.c^rtant        Important    ■  Opinion 

1967  6-42rM            7-50.6T'  "'  h'f"'^r>- 

1968  2-22.2%            5-55.6%  2-22.2%  -  ,  - 

1969  15-71.4%           5-23.8%  1-4.8% 

1970  10-43.5%           8-34.8%  5  23.7% 

TOTAL                  33-49.3%           25-37. T-l  3^11.9%  -  1-1.5% 


Present  PositiOT! 


1967 
1968 
1969 
1970 
TOTAL 


ExtreTnely 
Important 


Nd?iimallv 


4-28.6% 
2-22.2% 
14-66.7% 
9-39.1% 
29-43.3% 


Important  Important 


o-57.1% 
2-22.2% 
5-25.8% 
9-39.1% 
24-35.8% 


J -7.1% 
3-33.4% 
1-4.8% 
5-21.8% 
10-14.9% 


Not 
Important 


■2-22.2% 


2-3.0% 


No 
Opinion 


1-7.2% 

1-  4.7% 

2-  3.0% 


***********************************************•>***************«** 


■  During  the  Institute- 


Extremely 

Minimally 

Mot 

No 

Important 

Important 

Important 

Important 

Opinion 

1967 

4-28.6% 

9-64.  .3% 

1-7.1% 

1968 

4-44.4% 

4-44.5% 

1-11.1% 

1969 

9-42,9% 

11-52.4% 

1-4.7% 

1970 

8-3",.  8% 

8-.l4.8% 

4-17.4% 

3-13.0% 

TOTAL 

25-37.3% 

32-47.8% 

5-7.5% 

3-4.5% 

2-2.9% 

Present  Posit 

ion  , 

Extremely- 

Minimally 

Not 

No 

Important 

Important 

Important 

Important 

Opinion 

1967 

2-14.3% 

9-64.3% 

1-7.1% 

2-14.3% 

1968 

3-33.3% 

2-22.2% 

4-44.5% 

1969 

7-33.3% 

7-35.3% 

5-23.8% 

1-4,8% 

1-4.8% 

1970 

8-34.8% 

6-26.1% 

6-26.1% 

3-13.0% 

TOTAL 

20-29.9% 

2S-35.8% 

15-22.4% 

5-7.5% 

3-4.4% 

erJc 
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Question  lOK      Knowledge  gained  from  intcvaation  with  Institute  consultants 
..J  outside  of  the  classroom.    [During  the  institute],  for  the 

years  1967  and  1968  approximately  70%  of  the  institute  par- 
ticipants saw  this  as  being  minimally  important  to  important. 
During  th^  1969  and  1970  summer  institutes  approximately 
70-75%  of  the  trainees  saw  this  as  being  important  to  ex- 
tremely important.    From  their  [present  position],  for  the 
years  '67  and  '68  this  was  viewed  as  minimally  important 
to  important  by  40-45^^  of  the  participants .    During  the  '69  - 
'70  institute  it  was  viewed  as  important  to  extremely  important  by 
approximately  60-70^5  of  the  participants.    This  is  viewed  as 
a  very  positive  outcomx-.    The  desirability  of  pexsonal  con- 
tacts provided  by  the  institute  in  terms  of  continued  inter- 
action between  the  participants,  the  Bureau  staff  members, 
and  the  consultants  seems  to  bo  happening  with  a  rather  con- 
siderable amount  of  success,  particularly  as  a  result  of  the 
last  two  institutes . 

Question  lOL      Knowledge  of  proposal  writing  skills  acquired.    [During  the 
institute],  approximately  70-75%  of  the  trainees  viewed  this 
as  an  important  to  extremely  important  skill  to  be  acquired 
during  their  stay  at  the  institute.    From  their  [present  position], 
during  the,  years  ^67  and  '68  approximately  60%  of  the  participants 
saw  this  as  important  to  extremely  important  ability  in  their 
present  jobs.    This  percentage  has  moved  up  rather  sharply  in 
years  '69  and  '70  where  it  now  approaches  75-80%  of  the  response 
'       :  mode.    Again  indicating  that  school  systems  are  moving,  at  least 
with  these  participants,  in  the  area  of  educational  grantmansiiip. 


TABLE  iO  K 


During  the  Institute 


Extremely 

Miniinaily 

Not 

No 

Iniportant 

Important 

important 

Important 

Opinion 

J.  j\j  / 

■>  -  7   1 9^ 

/ - jU . U 1 

^- a1  .  4  (■ 

1  -  7  .  I'n 

1  Qf^  ft 

4"'iH  . 

1-] ] .2% 

xyoy 

i-14. 3-6 

:>-J.4..V:7 

1-4. y% 

1-4.8% 

iy/o 

6-20.1% 

12-52.2% 

1-4.4% 

1-4.4% 

i  VJiAL 

J.U-14.S-6 

12-17.9% 

5-7. S% 

4-6.0% 

Present 

Position 

ExtTemely 

Mot 

Mo 

Important 

Jmportant 

Important 

Important 

1967 

^  42.9% 

:-l4r3% 

3' 21.4% 

1968 

1-11.1% 

4-44.4% 

1969 

2-9.5% 

ii-52.4% 

3-14.  5^> ' 

4-iS.0% 

1-4.8% 

1970 

8-34 .8% 

1-4.4% 

1-4.4% 

TOTAL 

11-16,4% 

26-38. S^'a 

U-20,9% 

11-16.4% 

5-7.5% 

;  10  L 

During  the 

Institute 

Extremely 

Minimally 

Mot 

No 

Important 

Important 

Iniportant 

Important 

Opinion 

1967 

3-21.4% 

11-7S.6% 

1968 

5-55.6% 

2-22.2% 

1-11.1% 

1-11.1% 

1969 

6-28.6% 

12-57.1% 

1-4.8% 

1-4.8% 

1-4.8% 

1970 

.  9-39,1% 

9-39.1% 

4-17.4%  ■ 

1-4.4% 

TOTAL 

23-34.3% 

34-50.7% 

6-9.0% 

3-4.5% 

1-1.5% 

Present 

Position 

Extremely 

Miniinaily 

Not 

No 

Important 

Important 

lOToirtant 

IniDoa!:t;ant 

Opinidn 

1967 

3-21.4% 

6^*2.9% 

^-^.M  ■  ■ 

1-7.1% 

1-7 . 1% 

1968 

2-22.2% 

4^-44,4% 

5-^53.3% 

1569 

9-42.9% 

7-33.3% 

5-14. 1% 

1-4,8% 

1-4.8% 

1970 

12-52.2% 

9-39.1% 

1-4-4% 

1-4.4% 

TOTAL 

26-38.8% 

26-38-8% 

10-14.9% 

3-4.5% 

2-3.0% 

0 
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Question  lO.M  --  Importanae  of  veaditigs  in  required  texts  as  uell  as  outside 
;  sources. 

[During  the  institute],  rather  consistently  70-80%  of  the 
participants  viewed  this  as  being  minimally  important  to 
important.    There  is,  however,  some  pattern  of  an  increasing 
perception  of  this  activity  as  being  extremely  important  in 
lalfir  institutes,  '69. and  '70.    From  their  [present  position] , 
the  response  pattern  here  is  similar.    Approximately  70-80%  of 
the  participants  in  their  present,  positions  view  this  as 
minimally  important  to  important. 


TA3LE  10  M 
During  i;he  Institute 


1967 
1968 
1969 
1970 
TOTAL 


7-33:. 
4-17.4% 
15-22. 4?^ 


i'O 


liriDortant 

7-50.01 

S-5S.5% 

9-42.9% 

11-47.8% 

32-47.8% 


Minimally 

 I  iiii:-ortar.t 

4-28.6%  '~ 

18-26.9% 


Important 


1-4.4% 
Ul.5% 


No 
Opinion 


1-4.8'ji 
1-1.5% 


Present  Posi.tion 


1967 
1968 
1969 
1970 
TOTAL 


2-14, 


7-33.3% 
3-13,01 


Important 
~7r50%~ 
5-5.5.6% 
7-33.3% 
12-52.2% 
31-46.5% 


Mlnii.nally 
lirrDortant 

4-  44.4% 

5-  23.8% 

6-  26.1% 

lS-28,4% 


Not 
lifiportar.t 


2-  8.7% 

3-  4.  St. 


No 

Opinion 


1-7.1% 

1-  4.8% 

2-  3.0% 
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PAET  n 

PARTICmm  RESmSES  GROUPED  BY  PROFESSIOML  ROLES 
VALUE  OF  inSTRUCTIONAL  AREA  AS  PERCEIVED  DURING  THE  INSTITUTE 
AND  FROM  THEIR  PRESEHT  JOB  POSITIONS 

In  order  to  gain  further  insights  from  past  participants  as.  to  the 
value  of  the  instructional  areas  as  vioKed  -during  the  institute]  and  from 
their  [present  job  positions],  participant  responses  to  Question  10  were 
grouped  by  professional  roles.    That  is,  by  Teachers.  School  Administrators, 
State  Department  of  Education  Personnel,  and  Research/College  Personnel.  The 
reader  is  now  referred  to  Table  B  -  Participants  Responses  by  Professional 
Roles  in  Appendix  C  of  this  section  of  the  report. 

The  data  from  Table  B  was  then  condensed*  in  a  two  by  two  matrix 
and  subjected  to  a  statistical  analysis  using  Chi-square  testing  for  the  goodness 
of  fit  of  the  two  respondent  modes.    Because  the  cell  frequencies  were  small, 
the  Yates  Correction  for  Continuity  Factor  was  applied. 

The  purpose  for  grouping  the  participant  responses  by  professional  roles 
was  primarily  to  determine  whether  any  significant  shifts  had  occurred  by  pro- 
fessional roles  relative  to  the  importance  of  the  instructional  areas,  both  during 
and  after  the  institute.    Additionally,  the  resultant  data  could  also  be  utilized 
in  planning  for  future  institutes,  to  determine  the  adequacy  of  research  and/or 
evaluation  training  programs  geared  specifically  to  selected  roles  within  the 
educational  profession. 


TUL  ■^'■/^'f  •'■''^'"^  10        condensed  as  follows-  extremely 

LTS;;^41.  "P°'^^'"'^°  ^^P^^^^^^'-    ^"--^11^  -P-^-^  -d  no?  imporJLrt? 
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For  the  sake  of  simplicity,  illustrations  are  presented  below 
describing  the  comparisons  which  were  performed. 


TABLE  I 


TOTAL  GROW 


DURING 


AFTER 


TOTAL 


IMPORTANT 


NOT 
IMPORTANT 


TOTAL 


TABLE  II 


SCHOOL  VS.  OTHERS  -  DURING 


NOT 

IMPORTANT  IMPORTANT  TOTAL 

I  '  [  ; — '  ' — '  ' — • — -| 

SCHOOL  I 


OTHERS 


TOTAL 


ERIC 


NOTE:    School  refers  to  "teachers"  and  "school  administrators."  Others 

refers  to  "State  Department  of  Educational  Personnel"  and  "Research/ 
College  personnel." 
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TABLB  III 


SCHOOL  VS.  OTHERS  -  AFTER 


IMPORTANT 


NOT 
IMPORTANT 


TOTAL 


SCHOOL 
OTHERS 
TOTAL 


TABLE  n 


TEACHERS  Am  SCHOOL  ADMINISTRATORS 
DURING  AND.  AFTER 


ADMINISTRATORS 

§  TEACHERS-DURING 

ADMINISTRATORS 

,  §  TEACHERS-AFTER 


IMPORTANT 


NOT 
IMPORTANT 


TOTAL 


!  —  

TOTAL 
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TABLE  V 


STATE  DEPABTMEnr  OF  EDUCATION  FERSOmEL  AND 
■  COLLEGE/RESEARCH  PERSONNEL 


STATE  DEPT.  § 
COL/RES  —DURING 


STAT'"  DEPT.  S 
COL/utS  —AFTER 


TOTAL 


DURING  AND  AFTER 


IMPORTANT 


NOT 
IMPORTANT 


TOTAL 
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The  results  of  the  Chi-square  analysis  for  each  of  the  comparisons 
^^^jg  are  discu5Sed  below: 

I:  Of  the  thirteen  Chi -squares  reported,  significant  differences 

were  recorded  in  three  areas;  Question  lOB  —  the  aonstruation 
i^'^'i  identification 

the  skill  aoquired  in  uHlizing  the  Program  Evaluation  Review 
Technique  (PERT )^  and  (Question  IQJ      Knowledge  gained  from 
interaction  with  BERTS  staff  (exclusive  of  consultants). 

lOB-     For  the  total  group  the  shift  was  from  important  [during 
the  institute]  to  not  important  from  their  [present  job 
positions].    One  possible  interpretation  for  this  significant 
change  is  that  a  large  number  of  participants  are  not  in 
,   forinal  instructional  roles,  thereby  diminishing  the 
utility  of  learning  hierarchies. 

lOD-     for  the  total  group  the  shift  wa^  from  important  [during 
the  institute]'  to       important  from  their  [present  job 
.  positions] .    Perhaps  one  explanation  for  the  significant 
change  is  that  PERT  is  a  highly  sophisticated  tool  for 


For  the  total  group,  the  shift  was  from  important  [during  the 
institutej  to  not  important  from  their  [present  job  positions] . 
One  possible  explanation  we  can  off er  is  that  during  the  instit 
the  BERTS  staff  serve,  in  .a  sense,  the  role  of  middle  men  be- 

^  tween  the  consultants  and  tlie  participants.  ;  This  role  has  take 
the  form  of  providing  information  between  the  participants  and 
consultants  relative  to  goals  of  the  institute  and  related 
instructional  segments,  relating  to  various  levels  of  expecta- 
tains  for  both  groups,  etc.    Another  interpretation  could  be 
that  for  the  four  years  the  BERTS  staff  has  been  primarily 
responsible  for  instruction  in  statistics  and  computer  programm: 
In  Part  I  -  responses  to  Questions  10F,G,H  indicated  that 
participants  find  these  areas  less  important  in  their  present 

^  job  positions  than  during  the  institute.  As  such,  this  would  ^ 
seem  to  account  for  the  shift  in  opinion  relative  to  the 

knowledge  gained  from  the  BERTS  staff. 
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TABLE  II:  Of  the  thirteen  Chi-squares  reported,  a  significant  difference 

was  recorded  in  one  area:     Question  lOJ  -  knowledge  gained 
from  interaction  with  BERTS  staff  (exclusive  of  consultants) . 

Commentary: 

lOJ-    For  school  personnel  vs.  non-school  personnel  [during  the 
institute],  the  non-school  personnel  viewed  lOJ  as  not 
important.     This  is  to  be  expected.     Over  the  four  years 
the  participants  within  this  category  had  attained  a  reason- 
able level  of  awareness  of  the  instructional  content  prior 
-  to  attending  the  institute.     The  school  personnel,  however, 
viewed  Question  lOJ  as  important  during  the  institute. 


TABLE  III:         Of  the  thirteen  Ghi -squares  reported ,  significant  differences 
were  recorded  in  two  C23  areas:     Question  lOE  —the  CIPP 
model  of  Evaluation  (Context j,  Inputs  Process  &  Product); 
and  Question  lOJ  --  Knowledge  gained  from  interaction  with 
-  BEETS  staff  .  (exclusii)e  of  consultants) . 

Cormentary:  / 

lOE-    For  school  personnel  vs.  non-school  personnel  [after  the 
institute],  non-school  personnel  viewed  lOE  as  important . 
.  This  significant  change  could  perhaps  be  attributed  to  the 

fact  that  non-school  personnel  have  had  more  opportunity  to  employ 
the  concepts  of  CIPP.  ^ 


-40- 

lOJ-    The  pattern  is  similar  to  Table  II,  and  we  offer  the  sane 

interprc^;ation  with  one  additional  interpretation.    One  of  the 
Bureau's  major  functions  is  to  provide  various  services  to 
school  personnel  throughout  New  England.    As  such,  the  Bureau 
remains  in  constant  contact  with  school  personnel.     This  could 
perhaps  account  for  school  personnel  considering  Question  IQJ 
important . 


TABLE  IT 

DURING  H.E.W.   SUMMER  INSTITUTE 


TABLE  III 
AFTER  H. e.W.   SUMMER  INSTITUTE 


SCHOOL  VS. 

niHEPS 

QUESTION    10- A 

SCHOOL  VS. 

OTHERS 

QUESTION  IC- 

30  4 
22  5 
52  9 

34 
27 
6  1 

Not  Significant 

30  4 
19  7 
49  11 

34 
26 
60 

Not  Significant 

CHI   SQUaRE  = 

0.14 

CHI  SGUARE= 

1.36 

SCHOOL  VS. 

QTHE(^S 

QUE  S  HON  10-B 

SCHOOL  VS. 

OTHERS 

QUESTION  lO-i 

24.  9 
19  9 
43  18 

33 
2  8 
61 

L.    O  JlKlli-X  JLUcLJl  L 

18  15 
9  17 
27  32 

■  3  3 
26 
59 

inol  oigniricEiiL 

CHI  SQUARE= 

0.C2 

CHI   SOUARfi  = 

1.59 

SCHOOL  VS. 

OTHERS 

v-^UESTION  IC-C 

SCHOOL  VS. 

OTHERS 

QUESTION  IC-l 

28  5 
26  3 
54  8 

33 
29 
62 

Not  Significant 

24  9 
22  6 
46  15 

3  3 
28 
61 

Not  Significant 

CHI  SQUARE= 

0,03 

CHI   S GUAR E= 

0.05 

SCHOOL  VS. 

OTHERS 

QUESTION'  10-0 

SCHOOL   VS . 

OTHERS 

QUESTION  10- 

25  7 
19  8 
44  15 

3  2 

27  • 
59 

Not  bigniricant 

16  16 
15  11 
■  3  1        2  7 

32 
26 
58 

Not  Significant 

CH  I   S GU  AR  F= 

.0.15 

CHI  SQUARG= 

■  Q.  10 

SCHOOL  VS. 

OTHERS 

QUESTION  IC-E' 

SCHOOL  VS. 

OTHERS 

QUESTION  10- 

22  9 
24  2 
46  11 

31 
26 
57 

Not  Significant 

17  14 
41  17 

31 

on 

58 

Significant  At 
.02 

CHI  SQUAPE= 

2.88 

CHI   SQUAR  E  = 

6.5? 

SCHOOL  VS. 

OT  HER  S 

QUESTION  1C.-F 

SCHOOL  VS. 

OTHERS 

QUESTION  IC- 

22  13 
8  9 
30  22 

35 
17 
52 

Not  Significant 

15  21 

•    8  9 
■     2  3  30 

36 
17 

53 

Not  Significant 

QHI  SQU  ARE- 

0.61 

CHI  SOUARE= 

0.01 

ERIC 


^-^i.*^  H.cH.  su^.eR  iNsrnuTE  after  h.,.,.  su.vpp  i^.Tp-.   -  ' 

SCMnOL  vs.   HTH.RS  QUESTIO,VlO-G      SCHOOL  VS.  OTHFRS  n 

'^''^^''5  QUESTION  10- 


16 

19 

35 

5 

12 

17 

21 

31 

5  2 

CHI  SOUARe=  0.60 
SCHOOL  VS.  OTHERS 


29 

7 

36 

11 

9 

2!: 

40 

16 

56 

Not  Significant 

QUESTION  IC-H 
Not  Significant 


12 

22 

34 

5 

13 

1  8 

17 

3  5 

52 

CHI  SOUflRE-  0,06 
SCHOOL  VS.  OTHERS 


CHI  SOUARE=  ,  2,96 
SCHOOL   VS.  .OTHERS 


28 
14 
42 


8 
5 
13 


36 
19 
5  5 


CHI   SOUAP  E-  0.00 
QUESTION    lO-l'      SCHOOL   VS.   OTHERS  ^ 


34 

3 

37 

23 

6 

29 

57 

9 

66 

CHI  SQUARE=  1,25 
SCHOOL  VS.  OTHERS 


35 

1 

36 

19 

10 

29 

54 

11 

65 

CHI  SQUARE=  ^,34 
SCHOOL  VS.  OTHERS 


Not  Significant 


QUESTION  10-J 


.01 


OUESTI  ON   1  0-K 


32 

5 

37 

21 

7 

28 

Not 

53 

12 

6  5 

CHI  SOI)ARE=  0.74 
SCHOOL  VS.  OTHERS 


2  9 

7 

36 

At 

1  3 

15 

28 

4  2 

22 

64 

Not  Significant 

QUESTION  10-H 
Not  Significant 

QUESTION  lO-l 
Not  Significant 

QUESTION  10-J 


Significant  At 
.01 


CHI  SQUARE=  6.69 
SCHOOL VSe  OTHERS 


25 

9 

34 

22 

7 

29 

Not  Significant 

47 

16 

63 

CHI  SOUAR  e=  0.01 
SCHOOL  VS.  OTHERS 


18 

17 

35 

18 

9 

27 

36 

26 

62 

CHI  jSQlJAR  E=  n.90 
SCHOOL  VS.  OTHERS 


33 

3 

36 

23 

7 

30 

Not 

56 

10 

66 

CHI  SQUARE^  1.82 
SCHOOL  VS.  OTHERS 


QUESTION  IC- 
Not  SignifiVsmt 


CHI   SOUARE=         0.0  8 
QUESTION    IG-M    SCHOOL  VS.  OTHERS 


30 

7 

37 

21 

7 

28 

Not 

51 

14 

65 

27 
20 

O  47 

ERIC 


10 
9 
19 


37 
29 
66 


Not  Significant 


26 

11 

37 

17 

11 

28 

43 

22 

65 

QUE  ST  ION  IC-K 
Not  Significant 

QUESTION  10-L 
Not  Significant 

QUESTION  10-M 
Not  Significant 


•^T  SQUARE=      .  0.01 


CHI  SQUARE=  0.29 
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TABLE  IV:  o£  the  thirteen  Chi-squares  reported,  a  significant  difference 

was  recorded  in  the  area:    Question  lOD  -  tlie  skill  acquired 
^-     in  utilizing  the  Pvogvaxn  Evaluation  Review  Technvgue  (PERT). 

Commentary: 

lOD-    For  teachers  and  school  administrators,  the  shift  was  from 

important  [during  the  institute]  to  not  important  [after  the 
institute].     This  seems  to  support  what  the  total  group  felt  as 
reported  in  Table  I.    That  is,  teachers  and  administrators  view 
as  less  important  the  PERT  technique  in  their  present  job 
positions . 


TABLE  V; 


Cormentary : 


Of  the  thirteen  Chi-squares  reported,  one  significant  difference 
was  recorded  -  Question  lOB  -  the  construction  and  identification 
of  learning  hierarchies. 

For  state  department  personnel  and  college/research  personnel,  " 
the  shift  was  from  important  [during  the  institute]  to  not 
important  [after  the  institute].    Again,  this  seems  to  support 
the  total  group  felt  as  reported  in  Table. I. 


TEACHERS   AND  ADM  IN  ISTR  ATOPS 


30 

4 

34, 

30 

3^ 

60 

8 

68 

CHI   SQUARE^  0^.1^ 
TEACHERS   AND  ADMINISTRATORS 


24 

9 

33 

IB 

15 

42 

24  • 

6l 

CHI  SQUARE=  1.64 
TEACHERS  AND  ADMINISTRATORS 


28 

5 

33 

24 

9 

33 

52 

14 

66 

CHI  SQUARE=  0.82 
TEACHERS  AND  ADM  INISTRATORS 


25 

7 

32 

16 

41 

23 

64 

CHI   SQUARE=  4.34 
TEACHERS   AN  D  ADM  IN  I STR  A  TOR  S 


22 

9 

31 

17 

14 

31 

39 

23 

62 

CHI   SQUARE=  1.11 
TEACHERS  AND  ADMINISTRATORS 


22 

13 

35 

15 

21 

36 

37 

34 

71 

q'HI  SQUARE=  2.40 


TABLE  II 

DLRING   AND   AFTER  ,  yuEST  I  ON   l::-- A 

Not  Significant  ; 

OLRING   AND   AFTER  GUESTICN   lO-B  ' 

Not  Significant 

DLRING  AND  AFTER  CUESTIQN  10-C 

Not  Significant 

DURING  AND  AFTER  QUESTION  10-D 

Significant  at  .05 

DURING  AND  AFTER  QUESTIGN  10-E 

Not  Significant 

DURING  AND  AFTER  QUESTION  x 

Not  Significant 


STATE  DEPARTMENT  An  0'  COL  o /PE  S  , 


DLRI KG   AND  AFTER 


CUES!  ii. 


11 

9 

20 

5 

19 

25 

14 

39 

CHI  SQUAP.E=  0.78 

STATE   DEPARTMENT  AND  COL./RFS. 


23 

6 

29 

21 

7 

28 

44 

13 

57 

CHI   SQUARE=  0,01 

STATE  DEPARTMENT   AND  COL. /RE  5, 


19 

10 

29 

13 

15 

28 

32 

25 

57 

CHI  SOUARE=  1,40 

STATE  CEPARTHENT   AND  CQL./RFS, 


CHI  SOUARE=  0,22 

STATE  DEPARTMENT  AND  COL. /RES. 


23 

•  7 

30 

21 

7 

28 

44 

14 

58 

CHI   SQUAR  E=  0.03 
STATE  DEPARTMENT   AND  COL. /RES, 


20 

9 

29 

17 

11 

28 

37 

20 

57 

CHI  SQUARE- 


0.14 


ERIC 


Not  Significant 


DURING   AND  AFTER 


Not  Significant 


DURI  r\'G  AND  AFTER 


Not  Significant 


22 

7 

29 

18 

9 

27 

Not 

40 

16 

-56   

DURING  AND  AFTER 


Not  Significant 


DURING   AND  AFTER 


Not  Significant 


DURING  AND  AFTER 


Not  Significant 


QUEST  I  CN 


QUESTION   10 --J 


QUEST  I  Cr,  ] 


QUESTION  10-L 


QUESTION  l; 


27. 

5 

27 

19 

7 

26 

41 

12 

r)3 

TABLE  V 


Not  Significant 


CHI  SQUflRF;=  0'.16 

STATE  CEPARTMENT  AND  COL. /RES. 


DURING   AND  AFTER 


QUESTION , lU-A 


CUeSTION  10-B 


19 

9 

28 

9 

17 

26 

Significant  at  .05 

28 

26 

54 

'"-HI   SOUARE=  4.71 

STATE   CEPARTMENT  AND  COL. /RES, 


OUR  I  NO   AND  AFTER 


QUESTION  10-C 


26 

3 

2  9 

22 

6 

28 

Not 

48 

9 

57 

CHI   SOUARE=  0.61 

STATE  DEPARTMENT  AND  COL«/RES, 


CHI  SQUARE-  0.00 
STATE  DEPARTMENT  AND  COL. /RES. 


8 

9 

17 

8 

9 

17 

16 

18 

34 

CHI  SQUARE=  0.12 

STATE  DEPARTMENT  AND  COL. /RES. 


5  12 
O  13 

ERIC  ^5 

CHI  SOUARE= 


17 

18 
35 


Not  Significant 


DURING   AND  AFTER 


19     .  8 

27 

15  11 

26 

Not  Significant 

34  19 

53 

CHI  SQUARE= 

0,46 

DURING  AND  AFTi 

24  2 

26 

24  3 

27 

Not  Significant 

48  5 

53 

DURING   AND  AFTER 


Not  Significant 


DURING   AND  AFTER 


Not  Significant 


QUESTION  10-D  /, 


Question  io-e 


QUESTION  10-F 


QUESTION  10-G 


0,07 


TEACHERS   AND  ADMINISTRATORS 


OUR  I  KG   AND  AFTE^^ 


CUFSTIGN 


16 

19 

35 

12 

22 

34 

28 

^1 

69 

CHI    SOUARF=  0.40 
TEACHERS    AN  C  A  D:M  IN  I  S  TR  ATQR  S 


29 

7 

36 

28 

8 

36 

57 

15 

72 

CHI    SQUARE^  0^00 
TEACHERS   ANIJ  ADM  IN  I  STRATQR  S 


34 

3 

37 

32 

5 

37 

66 

8 

74 

CH[  SQUARE=         0.  lA 
TEACHERS   ANO  ADMINISTRATORS 


CHI   SQUARE=  3.52 
:T  EACHERS   AND  ADMIN  I  STRATDRS! 


25 

9 

34 

18 

17 

3  5 

43 

26 

69 

CHI  SQU/>RE=  2,71 
TEACHERS  ANO  ADM  IN  ISTR  ATQRS 


33 

3 

:  36 

30 

7 

37 

63 

10 

73 

CHI  SQUARe=  0.95 
^     TEACHERS   AND  ADMINISTRATORS 


ERIC 


;7 

10 

37 

26 

11 

37 

53 

21 

74 

S GUAR  6= 


Not  Significant 


OLR  r  NG   AND   AF  r 


Not  Significant 


OUR  INC   AND  AFTER 


Not  Significant 


DURING  AND  AFTER 


35 

1 

36 

29 

7 

36 

Not  Significant 

64 

8 

72 

DURING  AND  AFTER 


Not  Significant 


DURING  AND  AFTER 
Not  Significant 

DURING  AND  AFTER 
Not  Significant 


GlJESTION  10-H 


CUESTION  10-! 


QUESTION  lO-J 


CUESTION  10-K 


QUESTION  lO-L 


0*00 


EVALUATION  OF  THE  INSTRUCTIONAL  EFFECTIVENESS  OF  THE 
1967  to  1970  SUMMER  RESEARCH  TRAINING 
INSTITUTES  CONDUCTED  AT  THE 
UNIVERSITY  OF  NEW  HAf^PSHIRE 


SECTION  II 
RESULTS  OF  ON-SITE  EVALUATION 
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I  Rationale  for  Site  Visits 


The  second  major  evaluative  effort  focused  on  an  on-site  evaluation 
conducted  principally  by  Mr.  Charles  Williamson,  Jr»,  a  participant -observer 
in  the  1970  institute,  who  served  as  a  field  evaluator.    The  on-site  visits  were 
conducted  in  the  South  and  were  focused  primarily  on  a  number  of  the  1970 
institute  participants. 

The  rationale  for  conducting  the  on-site  follow-up  evaluation  was 
stated  as  follows: 

•  Will  enhance  the^'expertise'*  of  the  participants 

•  Will  establish  a  degree  of  rapport  between  the  1970  participant, 
the  Institute  personnel ,  and  the  LEA 

•  It  also  involves  the  LEA  directly  wi;'h  the  Institute  personnel 

-  LEA  personnel  and  participants  will  be  able  to  give 
a  candid,  yet  valuable  evaluation  of  the  program, 
vhich  in  turn  could  be  an  indication  of  the ' affective-  . 
ness  of  the  Institute  upon  the  immediate  needs  of  the 
agency.     (i.e.  promotions,  contributions,  of  partici- 
pants, etc.). 

-  Irvformation  could  be  obtained  by  the  Institute  staff 
for  OE  as  to  the  direction  of  future  institutes. 

-  LEA'S,  participants  and  future  participants  will  feel 

a  part  of  power  structure  and  will  be  willing  to  assist 
more  extensively. 

Approach  to  the  Qn-Site  Visits 

:    :  were  sent  by  the  field  evaluator s  to  ten  participants 

and  their  immediate  supervisors .    The  field  evaluator  then  followetj-up  by 
^^^^^^^^  ;t  requesting  permission 

\'yrr-:X^r-  ; f =  a  personal  inte3rview .    Al  1  participants  and  supervisors  honored  the  request 
ER^C     a-nd  time  and  locations  were  agreed  upon. 


The  on-site  visits  were  conducted  from  April  9,  1971  to  April  16, 
1971.    A  travel  summary  is  presented  below: 

TRAVEL  summi 

April  9,  1971  (D)  Athens,  Georgia- 

April  8,  1971  (D)  Thomson,  Ga. 

April  2,  1971  (A)  Mound  Bayou,  Miss. 

April  1,  1971  (A)  Clinton,  Miss. 

April  7,  1971  (D)  Knoxville,  Tenn. 

April  7,  1971  (D)  Chattanooga,  Tenn. 

April  14,  1971  (A)  New  Orleans,  La. 

(  April  15,  1971  (A)  Grambling,  La. 

April  16,  1971  (A)  Waco,  Texas 


Interview  Results       Participants  ^  Supervisors 

I  The  following  section  represents  results  of  the  on-site 

evaluation  as  reported  to  us  by  the  field-evaluator.  Copies  of  the  interview 
schedules  for  the  participants  and  supervisors  are  included  at  the  conclusion 
of  this  report . 


.  r a  r  t  i  c  i  r>an  t  G 

■  :^3rt'  I 

Professional  Personal 

1  .  .    .  --J      ■        '  ' 

There  was  no  d.if  f  er  enc  e   i.n   the   resronse?5   of  thc^  part  icipant  s 
as   to  the  reason   for  their   attendance  to  the   Institute,  I-^ 
vran  readily  recor-ai^ed  that  •  the  rerr^on   had  the  chance  to 
attend  the   program  .and    he       took   advantare   of  the  opportunity. 
The  averar«;e"  respon^:;e  was   reared   toward   their   nersoaal  reason 
as'  being  their  professional  reason.      The  ]ier:5onal  reason  in 
most   cases  vas  associated  vitii   their  professional  reason. 

2  *      a ,      Release  0 

h.      Transportation  3 

c .     finances  G 
Release  vras  not   a  prohlem  hecause  most  of  the  people  were  not 
employed   during  the   sunnier.  ^ 

Transportation  was   a  probler*  for  three  of  the  participants; 
"because  of  their  Tnode  of  transportation,      airline  schedules 
difficult  to  meet   and  their   livin.^?;  a  considerahle  distance  : 
from  the  airport . 

Finance,  'was   import ant  to  the   individuals   oecause  most  of 
\persohs'  had  to  read,1ust  their  lives   v  j  the   st ipend.  ■  However  > 
the  participants  were  reinhursed/'  for  their  travel  expenses^  the 
r^ajor  problem  was   centered  around  the  money 'earned  durin.r:  the;. 


these 


2/ 


Institute.  This  anovmt  vas  not  enourrh  to  aeet  trie  needs  of  tne 
faTxiliea   at  hone* 


3  .  iii;;;)!   rlxpectat  icnf: 

Employer  '  2 
Yourself  8 


Fam  ily 


9 


For  the  Record 
6 
1 
0 


Not  important 
1 
0 
0 


h»     Behavioral  objectives 

Learning  Ileirarchies 

Program/ Pro,^  GCt 
M.anagenent: 

PERT  ' 

CIPP 

Computer   Pro  grains 
Research .Techniques 
Social  Interaction 
ProDOsal  Wrltinn:  Skills 


1 


5'.  Excellent 

"Very  Good 
Good  . 
Fair 
:  Poor 


0 
2 


3 


3 
0 


V  :  *Kot  import  ant  2  .  ■ 

These  persons  lived  in  a  "black  community.  Their  schools  are 
not  involved  in  the  -Droblemr, . 


3/ 


The  supporting  response  was  alvays  "The  school  integration  pre 
gram  is  not  vhat   it   should  be". 


6. 


8. 


Problem  for  "black  teachers 
E(iually  competitive 
No  problem 
Not  applicable 

Favorable 

Unfavorable 

Indifferent 


Blacks 


Job 

For  supervision 

Under  supervision 

Social  Interaction 

Professional 
Activit  ies 


Southern  Northern 

8  . 


h 
8 


3 
2 
0 
3 

5 
1 
3 


Whites 


Direct  Decisi 


on 


8 
6 

Indirect  Decision 


Southern 
8 

3 
k 

a 


Northern 


8 


11/ A 


Most  of  the  decisiorls  are  made  through  the  supervisor.  Though 
there  are  two  individuals  who  make  decisions  directly,  they  also 
attempt  to  Influence  their  supervisor  and  make  decisions  for  their 
supordinates.  ' 


All  of  the  participants   indicated  thp.t  their  decisions  are 
usually  accept ed> 

10.  See  Item  1. 

All  of  the  participants   felt  that  -the   program  was   helpful  to  them 

11.  All  of  the  respondents  indicated  that  their  supervisors  felt 
that*  the  program  was   helpful   for  their  personnel. 


12. 


Promotion 
2 


Demo t  ion 
1 


No  change^- 
6 


13.  All  persons  vould  be  happy  to  attend  the  program  again, 
For  more         less  time. 

lU.     All  responses  vas  yes ♦ 


15^       Written  Directed  Project 

h  1 

Other  participants  (workshop,  etc.) 

a.     Federal           1  ' 

b ♦     State               2  I 

•    .        !  ■ 

G.     Local                6  ; 


N/A 

1 


16.  Useful 

17.  Share  Experience 


Nou  useful 


0 


N/A 
1 


Not   share  experience 


ERIC 


All  of  the  participants  shared  their  experiences  with  others 
through  conversations^  etc. 

Fart.  II 

1,     The  average  time   for  writing;  the  proposals  was   four  {h) 
veeks.     Other  joo   committments  cause  most  of  the  persons  the 
delay  in  returning  their  propos als  to  U.N.H* 

2»     Implemented  0 

Financed  1 

Accepted  in  your  area  h 

Negotiating  2 

Proposal  incomplete  2 


3.     Difficulty  '     No  difficulty  N/Resp. 

7  0  2 

k.     Accepting       —'y^'  Rejecting  N/R 

6    ,  1  2 

5*    :Edu?:ational  innovation  2 

Pro^ ect ed  ideas   ( new/revised )  3 

Community  concern  2  ' 

Educational  concern  1 

N /Response  1 


6*  ,  Financing  j. 
Completing  2 

TV   ; See ; Item. U.) 


8.  Institute  staff  6 
Participants  5 
Consultants  ; ...  0 
Other  2 

9.  All  response: 

Assistance  in  funding 

;10.      Indicated  ii^  iiarrative  Report. 

SUPERVISORS  RESPGilSSS 
1.     All  responses  were  excellent  (8) 

2  •      S am e  as  ab ov e     {  8  ) 

3  .     yes   (  6)       no^  ( 2 ) 
k.     Helpful  (8) 

5.  yes  (8) 

The  average  response  -   3  years 

6,  yes       8  responses 

T.       yes   (5)  no  (3)     •  - 

8.     yes       (8)  :  , 

All  of  the  promotion  depended  on  the  opening.  '  Only  two 
supervisors  indicated  a  specific  job.     They'^'were  promotions 
to  supervi  sor s  >  : 
/9  .  "  ■■^■■Supervisbr  -,  ;     1 /,V  \-       y-.-.;^  • 

'■■'.'■■■■Principal';     .''■  ,1  ' 

■'^Asst.'  ^Principal    '  ■;  :1'    ■  '  "■  '  :  '■■ 

Promotion  w/o  change  of  job  or  category  .  -  3  ;  : 


INTERVIEW  SOHSDIJLS 
Participant 


Part  1 

People  attend  Institutes  for  a  variety  of  reasons: 
(a.)  What  was  yoiir  pro.ressional  reason(s)  ? 
(b. )  What  vraa  your  major  personal  reason.{s)  ? 


What  problem  situations  did  you  encounters 

(a.)  Release 

(b)  Transportation 

(c.)  Finances 


3.      What  was  expected  of  you  after  your  return  from  tha  lnstituto> 
(a. )  from  your  employer 
(b.)  froiB  yourself 
(c«)  from"  your  faiaily 

^.  ar©  the  skills  and  knowledge  you  perceive  as  ess ential? 

5«      What  is  the  condition  of  school  integration  in  your  area  ? 
^"      ^<^^  is  it  afrecting  black  teachers  and  black  adiainistrators  ? 
7*      How  18  it  affecting  you  personally  ? 

8.      Swnepeple  believe  that  degrees  of  integration  betwe^k  i^it«a 

^^  ^^^^^^s^J^®  essential  as  ^  to 
J  such  as  the  Univei^ity  of  N«w  H«ajp 

■       ;: :thrpugh  th is  ;^lns t i:tute;.^'  ' contac:t3  :;;bstw««i : Wiv«r»e-' ^( 

seeking  cowmoR  to  a  panacea  for  biir  s be iil 

understand ing.    1  am  interested  in  the  experiences  of  social 

and  south 

0  ;vaiyijfig^;circuiBS'dmc«8;:'^'L 
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9.  Are  you  Baking  decisions  in  your  agency  ? 
A.  WhSit  are  they? 

3,  How  arc  Mi ey  accepted  ? 

10.  Do  you  feel  that  the  sumiaer  Institute  -A-as  helpful  to  you? 
<a.)  Kow? 

11.  How  does  your  supervisor  feel  about  your  attending  the 
Institute  ? 

12-    IIr.s  there  been  a  change  in  your  job  function  or  role  as  a 
result  of  your  x>sxticipation  in  the  Inatituta  ? 

13.    Would  you  like  to  attend  tho  Institute  again  for  six  weeks  ? 
(a*)  fcior©  or  Less  ? 

li^o    Wou3.d  you  return  if  you  recie?ved  a  degree  (i.e.)  Masters  _ 
or  Doctorate  in  Educational  Se search  ? 

15.  Have  you  written  any  proposals  or  directed  any  projects 
other  than  the  required  prf>osals  ? 

(a.)  Federal 
(b. )  State 
(c. )  I«ocal 

16.  Kow  useful  has  the  Institute  inforaation  be ©n  in  your  job  ? 

17.  Has  your  supervisor  invited  you  to  share  your  experiences 
with  ^others ,■ 
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How  long  did  it  take  you  to  write  your  proposal  ? 

Has  your  proposal  bv^en* 

(a)  iispieinentad 

(b)  Financed 

(c)  accepted  in  your  area 

Did  you  have  difficulty  getting  tho  program    fund&d  7 

HoK  are  your  superiors  accepting  tiie  idea  of  your  proposal? 

What  do  you  feel  are  the  succesaf^il  factors  of  your  proposal  ' 

With  what  aspects  of  your  proposal  air you  experiencing 
most  difficulty  ? 

Which  instruction  area  was  most  helpful  to  you  in  the  develop 
ment  of  your  proposal  ? 

Have  you  reciaved  any  assistance  froins 

(a)  Institute  staff 

(b)  participants  < 
(o)  Consultants 

(fi)  Other* 

What  is  your  basic  need  in  the  fulfiilroent  of  your  project  ? 

The  Observation  of  the  pro 40c t/Sr  any  consultative  assistance 
that  the  participant  m^^y^  kave* 


IRTEir/USVf  SCKgDlTL-g 
Supervisor 

What  -type  cf  employee  is  (Name  of  participant) — ,  

•o      What  is  your  evaluation  of  ' 

5.      Were  you  aware  of  His/her  attending  the  HEW  Institute  ? 

>.      Do  you  fQol  that  the  HEW  Institute  was  holpful  or  hiaraful  7 

:>•.      Do  you  utilize  ■  in  the  decision  making  l^rocesa  ? 

How  long  ? 

Is  he/  her  an  asset  to  your  program  ? 
7»      How  do  you  best    use  his/her  skills  in  thkn  agency  ? 

y 

Do  you  feel  that  he/she  will  earn  a  promotion    in  the  near 
future  ?    What  ?  Why? 

9.      Wh«n  was  his/her  last  promotion  ?    What  t 

10-    tiive  any  additional  information  about   ,   


APPENDIX  C 


EVALUATION  OF  THE  INSTRUCTIONAL  EFFECTIVENESS  OF  THE 
1967  to  1970  SUIillOR  RESEARCH  TRAINING  INSTITUTES 
CONDUCTED  AT  THE  UNIVERSITY  OF  NEW  HAMPSHIRE 


STATEiMENT  OF  OBJECTIVES 
1967  -  1970 


Objectives  -  USOE  Institutes  1967-70 


For  the  Institutes  that  occurred  in  1967  and  1968  the  objectives 
were  listed  more  or  less  generally  and  not  sub-divided  into  specific 
objectives. 

The  proposed  objectives  for  the  1967  Institute  were: 

The  development  of  a  piMe  -  ■  tpeveonal  reseavch  which  has 
reliivanoy  in  the  person     home  school  d-istrict  or  heme  state; 

The  studies  of  methods  of  educational  research  with  attention 
to  statistical  analyses  hy  which  the  hypotheses  are  either 
accepted  jr  rejected; 

The  us^  of  modem  data  processing  equipment  to  facilitate  the 
enaodiing  and  capturing  of  this  rese 

The  identification  of  present  research  literat^Are  applicable 
to  the  research  problems  of  the  trainees  home  districts. 


The  proposed  objectives  for  1968  were: 

The  stiidy  of  the  problmi  inherent  in  evaluating  the 
particular  educational  problem  with  which  the  trainee 
is  conccimed; 

The  study  of  major  alternatives  open  to  the  educator 
in  terms  of  educational  research  methodologies;  for 
instance^  the  [ experimental  approach  as  typified  by 
the  work  of  Campbell  and  Stanley;  or  the  context^ 
inputs \process  and  product  model  (CIPP)  as  presented 
by  Stufflebeam.    In  either  model  approach'^  the  program 
evaluaiion  and  review  technique  (PERT)  presently  being 
developed  for  education  by  Cook,  will  be  advocated  as 
the  method  of  organizitig; 

The  study  of  communication  techniques  applicable  to  the 
proper  implementation  of  the  decisionmaking  process  at 
various  levels  of  the  educational  system; 

The  use  of  modem  data  processing  equipment  to  fad  litate 
the  encoding  and  utilization  of  research  data; 

The  reading  ^  o^  literature  with  emphasis  on  the 

application  of  literature  relevant  to  the  research 
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Beginning  in  1969  a  general  proposed  objective  was  used  f ol lowed 
V  more  specific  objectives  as  can  be  seen  from  the  following  list : 


The  study  cf  the  problems  inherent  i.n  evaluating  the 
particular  ciirriQulim  changes  with  whiah  the  trainee 
is  oonaemed.    The  study  of  the  problem  of  evaluation 
Idas  approached  by  having  the  trainees  identify:,  write y 
and  evatiiate  objectives  for  ins triictional  programs  in  the 
areas  of  ourricultm  change  in  which  they  ware  interested. 

SPECIFIC  OBJECTIVES: 

At  the  end  of  the  program  the  participants  demonstrated  their 
ability  by  performing  the  following  tasks: 

a.  Identify  evaluative  techniques  which  can  be  utilized  to 
provide  information  for  making  decisions  about  curriculum 
change; 

b.  Identify  the  objectives  for  a  specific  program  in  their 
area  of  interest; 

c.  Distinguish  between  well -written  and  poorly-written 
objectives; 

d.  Identify  and  construct  performance  objectives; 

e*      To  translate  (when  possible)  into  performance 
objectives  the  objectives  stated  in  the 
curriculum  guides  presently  used  in  their  school 
system  which  are  stated  in  non-verbal  terms . 


The  study  of  the  major  alternatives  open  to  the  educator  in 
terms  of  educational  research  methodologies.    For  instance 
thejjexpeioimen^  by  the  work  of 

'  r  '':^Campbett  and  Sta^iley^  or  the  context ^  inputs  process  and 
product  (CIPP)  model  as  presented  by  Stufflebeam.  In 
either  approach  t}ie  program  evaluation  and  review  technique 
(PERT):,  developed  for  education  by  Cookj  will  be  advocated 
as  the  method  of  organizing  the  project. 

SPECIFIC  OBJECTIVES: 

At  the  end  of  the  program  the  partici;  its  demonstrated  their 
ability  to  perform  the  following  tasks : 

a.      Identify  the  classification  scheme  of  the  CIPP 
evaluation  model; 


b.  Construct  a  research  study  using  either  context 
evaluation,  input  evaluation ,  process  evaluation, 
or  product  evaluation; 

c.  Describe  the  differences  between  an  experimental 
and  quasi -experimental  design  for  educational 
research; 

d.  Order,  with  the  PERT  technique,  the  planning  of  an 
educational  research  study. 


The  study  of  oorrmuniaation  techniques  apptiaahte  to  proper 
implementation  of  the  decision-making  process  at  various 
levels  of  the  educational  system. 

SPECIFIC  OBJECTIVES: 

At  the  end  of  the  program  the  participants  demonstrated  their 
ability  to  perform  the  following  tasks: 

a.  Interpret  the  research  findings  of  several  journal 
articles; 

b.  Demonstrate  the  competency  to  apply  tfe  i:indings  of  a 
research  study  to  ones  local  school  situStion; 

c.  Describe  the  problems  associated  with  diss^^^mination 

and  adoption  of  the  general  kind  of  educatl©i2a2  reseaiC'h 
findings  to  a  local  school  system. 


The  use  of  modern  data  processi^^^^^^^  to  facilitate 

the  encoding  and  utilization  of  research  data. 

SPECIFIC  OBJECTIVES: 

At  the  end  of  the  program  the  participants  demonstrated 
their  ability  to  perform  the  following  tasks : 

a.  Construct  a  sample  computer  program; 

b.  Identify  the  problems  associated  with  designing  an 
optically  scanable  document; 

c.  Demonstrate  the  ability  to  operate  a  remote  terminal; 

d •      Name  and  describe  the  use  of  a  variety  of  modern 
data  processing  equipment  and  its  utilization  in, 
educational  research . 
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TJiB  veading  of  current  eduaational  literature  retevani  to 
the  reseoTGh  project  with  which  the  trainee  is  involved. 

SPECIFIC  OBJECTIVES: 

At  the  end  of  the  program  the  participants  demonstrated  their 
ability  to  perform  the  following  tasks: 

a.  To  identify  the  major  reference  source  for  educational 
research  literature; 

b.  Construct  a  bibliography  in  the  area  of  the  trainee's 
interest; 

c.  Distinguish  the  major  components  of  a  piece  of  w«jI1 
written  educational  research. 


The  proposed  objectives  for  the  1970  Institute  were: 

The  study  of  the  problems  inherent  in  the  management  of  an 
educational  evaluation  project. 

SPECIFIC  OBJECTIVES: 

At  the  end  of  the  Institute  the  participants  will  demonstrato 
their  ability  to  perform  the  following  tasks: 

a.  Define  systems  analysis  and  the  listings  of  tiia  basic 
steps  related  to  system  analysis  procedures; 

b.  Define  the  meaning  of  management  information  systems 
p^IS),  list  the  major  component  parts  of  an  educations 
information  maiiagement  system,  and  define  the  essential 
steps  in  the  implementation  ox  an  MIS  in  actual  practice ; 

c.  Define  program,  planning,  budgeting  systems  (PPBS)  and 
list  the  component  parts  of  the  PPBS  system; 

d.  Defiiie  program  evaiuation  and  review  technique  (PERT) , 
V     list  the ;basic  elements  of  ;  PERT  and  prepare  a  PERT 

network  from  a  simulated  problem  situation. 


si^icf?/  o/"        pro&Zema  inherent  in  evaluating  the  particular 
<r}^'^'^^^  ohanges  witli.  xihich  the  trainee  is  oonoernM.  The 
■}...  Study  of  the  problem  of  evaluation  will  be  attacked  by  having 
the  trainees  identify,  write and  evaluate  objectives  for 
instvuationa I  programs  in  the  area  of  ourrieu lum  change  in  which 
they  ore  interested. 
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SI^EClFtc  OBJECTIVES: 


At  the  end  of  the  Institute  thfi 'participants  will  demonstrate 
their  ability  to  perform  the  f 01 lowing  tasks: 

0-.      Iden't if y  the  objectives  for  a  specific  program  in 
the  area  of  their  personal  interest; 

t*'      Oisc^fminate  between  well-written  and  poorly-written 
behavioral  stated  objectives; 

C.      Identify  and  construct  a  set  of  performance  objectives; 

*d.      To  translate  (whore  possible)  into  performance,  objectives 
stated  in  the  abstract  that  they  prepared  as  applicants 
for  the  Institute, 


!f he  stiifiy  of  majo:p  alternatives  open  to  the  educatoT  -tn  terma 
of  eduQafif^ricil  v  es  ear  oh  methodologies .     For  instance^  the 
^^erifneni^l  appx'oach  as  typified  by  Campbell  and  Stanley^  or 

content ^^i^rput^  P^ooess  and  product  (CIPP)  model  ae  presented 
py  Stuff tebeam. 

SPECIFIC  OBJECTIVES: 

M  the  en<^  of  the  Institute  the  participants  will  demonstrate 
their  ability  to  perform  the  following  tasks: 

^.      Identify  the  classification  scheme  of  the  CIPP 

evaluation  model;  ■ 

\)'      9^nsti'uct  a  research  study  using  either  context  evaluation, 
input  evaluation 3  process  evaluation  or  product  evaluation; 

(J.      J^'Ssci'ibe  th^Li  difference  between  an  experimental  and  a  - 
q/Jasi "Experimental  design  for  educational  research. 


H^  st^idy  Of  the  problems  associated  h>ith  the  development^ 
,      d'i^^sem^nai>1^on  and  adoption  process  in  the  area  of  educational 

At  the  end  of  the  Institute  the  participants  will  demonstrate 
tmiT  abilities  to  perform  the  following  tasks: 

\i(?t.  f^iplicabie .    Late  funding  did  not  provide  sufficient  tims  for 

gt^plicants  to  Pi"epare  tHid  submit  research  proposal  abstract : 

^t?Stf a-ct  TequiJ^ement  deleted  from  trainee  recruitment  and  selection 


a.      Identify  the  problems  associated  with  designing  a  good 
research  evaluation; 

Construct  a  research  study  which  reflects  understanding 
of  research  design  and  measurement  theory; 

c.  Distinguish  between  interpretations  which  are  in  fact 
justified  from  the  data  and  those  which  are  not  justi- 
fied from  the  data; 

d.  Demonstrate  through  the  use  of  the  systems  approach  the 
problems  associated  with  developing,  disseminating  and 
adopting  the  results  of  educational  reseaivh  within  an 
educational  environment 


T?2e  study  of  aorrminiaation  techniques  appH^         to  the  proper 
implementation  of  the  decision-making  process  at  various  levels 
of  the  educational  systems. 

SPECIFIC  OBJECTIVES : 

At  the  end  of  the  Institute  the  participants  will  demonstrate 
their  abilities  to  perform  the  f^ollowing  tasks: 

a.      Interpret  the  research  findings  of  several  journal 
articles; 

b -      Demonstrate  the  competency  to  apply  the  findings  of  a 
research  study  to  one's, local  school  situation; 

c.      Describe  the  problems  associated  with  dissemination  and 
adoption  of  the  general  kind  of  educational  research 
findings  to  a  local  school  system. 


The  use  of  modem  data  proaessing  equipment  to  faoi  litate  the 
enooding  avd  utilization  of  research  data. 

SPECIFIC  OBJECTIVES: 

At  the  end  of  the  Institute  the  participants  will  demonstrate 
their  ability  to  perform  the  following  tasks: 

a.  Construct  a  simple  computer  program; 

b.  Identify  the  problems  associate:^  vith  designing  an 
optically  scanable  document; 


c.  f^emonstrate  the  ability  to  operate  a  remote  terminal' 

d.  Name  and  describe  the  use  of  a  variety  o£  modern  data 
processing  equipment  and  its  utilization  in  educational 
research. 


The  reading  of  current  educational  literature  relevant  to  the 
researah  project  with  which  the  trainee  is  involved. 

SPECIFIC  OBJECTIVES : 

At  the  end  of  the  institute  the  participants  will  demonstrate 
their  abilities  to  perform  the  following  tasks: 

a.  To  identify  major  reference  sources  for  educational 
research  literature; 

b.  Construct  a  bibliography  in  the  area  of  the  trainee's 
interest ; 

c.  Distinguish  the  major  components  of  a  piece  of  well- 
written  educational  research. 


PARTICIPANT  FOLLOW-UP  EVALUATION  QUESTIONNAIRE 
H.E.W.  SUIvlMER  RESEARCH  INSTITUTES  1967-1970 
U.S.  OFFICE  OF  EDUCATION 
NATIONAL  CENTER  FOR  EDUCATIONAL  RESEARCH  AND  DEVELOPMENT 


Prepared  by 

Bureau  of  Educational  Research  and  Testing  Services 

P.O.  Box  Q 
University  of  New  Hampshire 
Durham ^  N.H. 


March  4,  1971 


How  many  programs/projects  proposals  did  you  initiate/write  prior  to 
attending  your  Institute?   


How  many  programs/project  proposals  did  you  initiate/v/rite  since  attending 
your  Institute?  : 

Since  attending  the  Institute  have  you  written/ initiated  any  o£  the  following 
Proposals  (Fed.  /  State,  Local)  titles  .  


Evaluations: 


Research  Studies: 


My  job  role  and/or  function,  since  attending  the  Institute  has  changed  from 

IVhat  is  the  present  status  of  the  program/project/proposal  you  were  required 
to  submit  as  part  of  your  Institute  experience? 


Since  your  Institute  experien:;e  have  you  'been  in  contact  with : 

A)     Institute  Staff  YES      NO  B)     Institute  Consultants  YES 

C)    Other  Participants    .    YES  NO 

If  yes,  please  specify  the  context  (s)  in  which  this  contact  was  made 


Since  attending  the  Institute  have  you  been  called  ojv  to  conduct  in-service 
traiRing  prbgramsV  workshops  or  simply  give  presentations  in  areas  the 
Institute  concentrated  on? 

YES  NO  NOT  APPLICABLE       (If  yes,  please  specify  the 

number,  content  §  audience  of 
such  activities) 


8-    Have  you  taken  or  been  involved  in  courses,  seminars,  workshops,  etc,,  since 
your  Institute,  related  to  educational  research  and  evaluation? 

YES  NO 

If  yes,  please  specify  date,  title^  place,  etc. 


9.    What  have  you  made  of  the  instructional  materials  presented  to  you  at  your 
Institute? 


10. 


Please  rate  the  following  Instructional  areas,  as  presented  in  your  Institute, 
ACCORDING  TO  THEIR  IMPORTANCE Institute  and 
(b)    to  your  present  job  position. 


Developing 
and  writ- 
ing 

behavioral 
objectives 


The  con- 
struction 
and  identi- 
fication of 
learning 
hierarchies 


Under- 
standing the 
various 
iF-i^9^LStra- 


Extremely 

Important  Important 


Minimally  Not  Important  No  Opijix: 
Important      At  All  Response 


a  i 


Extremely                           Minimally      Not  Important      No  Opinion 
Important  -   Important      Important      At  All*  R^^nonse 

The  skill 
acquired 
in  util- 
izing the 
Program 
Evaluation 
Review 
Technique 
(PERT) 

I 

i 

J 

The  CIPP 

model  of 

Evaluation 

(Context, 

Input, 

Process , 

Product) 

a 

i 

.    -  . 

b 

The  skills 
and 

practice 
acquired 
by  writ- 
ing 

computer 
programs 

a 

b 

■  • 

- 

■ 

: 

■ 

.  - 

.  ■ 

The  skills 
and 

practice 
acquired 
in  operat- 
ing 

computers 
and  computer 
supportive 
hardware 

a 

• 

■ 

—  mm 

b 

■ 

■ 

- 

- 

• 

i 

i 
1 

i 

i 

I 

■ ..  ,| 

■  I 

1 

The 

statistical 
skills 

of fered  as  an 
introductory 
base  or 

general  review 
of  those 
needed  in 
research 

a! 

-  i 

i 
\ 

—  — t 

b 

,  -  ..  ..  1 
■  f 

-  ■! 

s 

,  —   

-— — — — 



Knowledge 
gained  from  .  : 
interaction 
with  fei low  ^ 
participants 

a 
b 

Extremely  Minimally      Not  Important      No  Opi'.: 

Important      Important      Important      At  All  Response 


Knowledge 
gained 
from  inter- 

a 

i        ^  — - 

i 

B.E.R.T.S. 
staff  ; 
(exclusive 
of  outside 
consultants) 

b 



Knowledge 
gained 
from  inter • 
action  with 
Institute 
consultants 
outside  of 
the  class- 
room 

a 

b 

_ 

Knowledge 
of  proposal 
writing 
skills 
acquired 

a 

Importance 
of  readings 
in  required 
texts  as 
well  as 
outside 
sources 

a 
b 

.   1 

11.    Do  you  feel  that  your  job  function  or  role  has  changed  as  a  result  of  your 
participating  in  the  Institute?  ' 

/~~7      very  significantly  significantly  /~7    some  what 

not  at  all  /    /    don't  know 


12.  Would  you  commit  yourself  to  a  number  of  such  Institutes  (6 -weeks)  to 
receive  an  advanced  degree  such  as  a  Master 's  in  Educational  Research 
and/ox  Education? 

:  YES  /~/      NO  /I7  MAYBE 

13.  Participants  to : Institutes  of  this  type  should  be  grouped  by  experience, 
job  function  and  expertise  on  a  more 

ERJC^— /      Homogeneous  basis         /  /    Heterogeneous  A^/     no  opinion 


14.  :  Do:  you  feel  .the  use  ^^^^^^^      name*'  consultants  from  large, : outstanding 
\;      universities  at  these  Institutes  is 

//:  cf  extreme  value       y/"  som  be  handled  by  other  less 

;  /    ■  _  we ll_kno\^  but  comparable  trained  personnel 

;      7/  of  no  significant  value      fj  no  opinion 

15.  :   What  are  some  of  the  changes  you  would  make  assuming  vou  were  conducting 

the  Institute?  "~ 


16/   What  do  you  believe  the  greatest  strengths  of  such  Institutes  are: 


17.  '  V/hat  do  you  believe  the  greatest  weaknesses  of  such  Institutes  are : 


Additional  comment  or  personal  opinion; 


Complettj  any  statements  on  back 
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PARTICIPANT  JOB  CHANGES 


1967  -  1970 
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ARE  SUf^D^lER  INSTITUTES  A  VIABLE  INSTRUCTIONAL  STRATEGY? 

In  this  section,  we  depart  somewhat  from  the  statistical  treatment 
of  our  data  in  order  to  present  in  capsule  form,  our  response  to  ttie  ques 
Are  Summer  Research  Trainlfig  Institutes,  as  typified  by  those  conducted  art 
the  University  of  New  Hainj.>hire,  a  viable  instroetional  strategy? 

The  participants  point  of  view  generally  seems  to  suggest  that 
summer  institute  programs  have  been  reasonably  effective.     We  would  also  agree, 
But  we  must  hastily  add  that  agreement  does' not  necessarily  constitute  total 
acceptance  of  the  designs  of  past  institutes.     Based  on  our  experiences  over 
the  last  four  years,  some  suggested  modifications  are  worth  noting. 

First  of  all.  it  is  imperative  that  the  individual  who  designs  any 
such. institute  actually  directs  it .    Not  only  must  he  oversee  the  actual 
operation  of  the  institute,  he  must  also  invest  considerable  time  and  effort 
in  the  months  prior  to  the  start  of  the  institute.     As  CIRCE  points  out  in 
their  Final  Report,  -'to  do  an  effective  job  of  rumiing  an  institute,  the 
director  must  be  free  from  the  entanglements  and  responsibilities  of  his 
regular  day-to-day  duties.    As  long  as  the  director  remains  physically 
proximate  to  Ms  'home  institution'  he  tyiU  find  it  very  difficult  to  avoid 
those  entanglements.    Sudden  emergencies,  unexpected  complications  occur 
too  frequently^partiaularly  in  a  university  set  ting- for  us  to  be  able  to 
assume  that  a  'paper  commitment'  to  an  institute  will  in  fact  hold  up  on  an 
assumed  100%  basis.    The  task  of  the  adjninistrative  staff  of  such  an 
institute  is  enormous... Logistics,  timely  funding,  communication,  facilita- 
tion, interpersonal  skills,  administrative  know  how  are  all  essential." 
Institutes  organized  around  the  concept  of  utilizing  consultants  for  weekly 


instructional  segments,  compound  the  need  for  coordination,  transition 
from  week  to  week  and  organization  to  an  almost  impossible  degree.'  "The 
demands  such  an  approaches  places  on  staff,  consultants,  and  participants 
are  hard  to  meet  (CIRCE)."      It  seems  unlikely  therefore,  that  an  institute 
director  can  be  effective  as  long  as  his  responsibilities  divert  him  from  the 
actual  directing  o:f  the  institute  program. 

With  respect  to  the  selection  of  participants,  the  notion  that  a 
summer  institute  can  serve  the  needs  and  desires  of  a  cross-section  of 
professional  educators  is  perhaps  somewhat  unrealistic.    The  participant 
response  data  described  in  the  report- on  the  heterogeneity  or  homogeneity  of 
participant  grouping  is  rather  inconclusive.     However,  "an  alternative 
to  be  considered  is  a  series  of  narrowly  formed,  short  or  long-term  institutes 
or  workshops,  each  serving  a  homogeneous  group  of  participants-homogeneous 
with  respect  to  dob  responsibilities,  educational  backgrounds,  perhaps  even  ^ 
regional  locations."  CCIRCE) 

Related  to  the  area  of  selection  is  the  issue  of  recruiting 
^ethnic-minority  groups.    The  use  of  minimum  participant  quotas  and  pre-institute 
orientation  for  ethnic-minority  groups,  as  required  by  the  1970  Institute  is 
viewed  as  being  unnecessary  and  in  many  instances  perceived  by  these  groups  as 
mere  ■■tokenism'. .    Wo  do  feel  however,  that  institutes  of  the  type  and  quality 
at  U.N.H.  in  the  summer  of  1970  offer  a  great  deal  to  these  groups ,  and  partici- 
pants representing  various  minority  groups  ought  to  be  actively  recruited. for 
future  institutes. 
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With   -pecific:  xegard  to  the  pre-iimstitute  orientation  for  the  1970 
Inssamw.,  one  mack  participant  suimned  it  tap  very  well,  "we  have  had  to  adjust 
to  aur  ^■■.ir>om,:-rb  and  changes  in  our  situations  all  our  lives;,  we  adjust 
easier  tAan  am^  else,  we  have  had  more  practice-^so  why  select  only  us 
to  .mittemd  the  'pMs-session?"  ■  ' 

ilf  otto  summer  institutes  are  organized  around  the  concept  o£  bring- 
ing Sis  '^expert'-zconsultants  for  relatively  short  periods  o£  time  for  selectsd 
instrnictional  areas,  an  elaborate  and  detailed  external  and  internal  "monitoring 
system-  should  be  required.     Such  a  system  is  expensive,  howev-ur,  it  is 
vitally  necessary,  especially  if  heterogeneous  participant  groups  are  to  be 
involved. 

From  our  point  of  view,  one  of  the  most  interesting  dimensions  of  the 
four  summer  institutes  has  been  lack  of  a  clear  definition  distinguishing 
whether  ti7e  training  programs  have  been  geared  predominantly  to  train  educa- 
tional evaluators  or  educational  researcher's.     For  some  people,  this  issue  could 
be  perceived  as  trite  and  discounted  as  a  problem  in  semantics.     For  us,  the 
distinction  is  significant  and  worth  noting.     Looking  back  over  the  four  years 
at  the  instructional  content  of  each  institute  program,  instructional  areas 
seem , to  be  reflecting  a  synomous  interpretation  of  research  and  evaluation, 
(i.e. ,  Campbell  and  Stanley.  Quasi-Experimental  Design,  Statistics  and  Computer 
:  Programming  vs.  CIPP,  Adoption,  and  Diffusion  of  Innovation,  Program/Project 
Management)  the  distinction  here,  should  not  bo  construed  as  evaluating  the 
relative  value  of  research  as  opposed  to  evaluation.    Our  purpose  is  to  simply 
suggest  that  id:thin  the,  design  framework.  .o£  summer  institutes,  consideration 
shcHiM  IbK  givesm.  to  who  are  we  training  raicd  iwhy . 
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Today,  there  is  a  growing  emphasis  on  what  the  public  dollar  is 
purchasing  in  terms  of  educational  serx^ices,  products  and  quality.     In  short, 
accountability.    There  can  be  little  debate  that  evaluation  is  a  critical 
element  in  any  attempt  at  accountability.     The  evaluator,  armed  with  the  tools 
of  his  profession  and  skills  acquired  through  training,  can  contribute  signifi 
cantly  to  the  improvement  of  the  quality  of  education  a  school  system  offers 
as  well  as  to  the  cost  efficiency.     IVith  money  becoming  scarce  in  many  school 
districts  the  need  does  not  appear  to  be  one  of  training  additional  personnel 
to  add  to  an  educational  bufeaucracy  but  to  train  evaluators  from  selected 
personnel  already  in  the  system         professionals  capable  of  evaluating  the 
effectiveness  of  their  own  (er  other)  programs  within  the  system. 

This  is  not  to  say  that  there  is  not  a  need  for  educational 
researchers.     Educational  researcliers  have,  in  the  past  and  undoubtedly  will 
continue,  to  contribute  significantly  to  educational  change. 

Ideally  then,  we  should  be  offering  institutes  to  train  each  of 
these  groups  of  professionals.    However,  if,  as  so  often  is  the  case,  we 
are  constricted  by  time\ or  fiscal  limitations  into  making  an  either/or  decision 

we  will  be  forced  to  examine  very  closely  the  priority  list  of  educational  : 

needs  and  select  the  group  most  capable  of  fulfilling  those  needs . 


SUGGESTED  MODIFIED  INSTITUTE  MODEL 


TIME 

This  model  would  consist  of  two  distinct  four  week  instructional 
segments  each  summer.     Each  segment  would  focus  on  one  specific  area,  such  as 
program  management,  or  evaluation  models,  etc.,  in  the  case  of  evaluation;  or, 
for  research  they  could  be  experimental  design  or  statistical  analysis.  Each 
segment  would  have  appropriate  secondary  supportive  instructional  areas 
offered  on  alternate  days   (statistics,  computer,  change  process,  educational 
grantsmanship,  etc).     Each  instructional  segment  would  offer  six  credit  hours,  s 
by  going  to  both  4  week  segments  in  the  summer  a  participant  would  earn  twelve 
credit  hours.     The  entire  program  would  consist  of  three  consecutive  summers 
or  a  total  of  thirty-six  credits  and  the  awarding  of  a  Master's  Degree  in 
Educational  Research  or  Educational  Evaluation. 

A  participant  not  interested  in  the  total  program  or  the  degree 
couldenter  at  any  level  and  remain  as  long  as  he  chose.     Participants  with  a 
proven  expertise  in  one  secondary  supportive  instructional  segment  (such  as 
PERT)  would  be  given  credit  for  that  expertise  and  would  not  have  to  attend  that 
particular  s,egment . 

A  four  day  instructional  week  might  prove  beneficial  by  allowing 
people  who  normally  might  not  be  able  to  attend  due  to  time  constraints  to 
do  so.    For  example,  superintendents,  state  department  personnel,  etc.,  could  • 
attend  the  institute  four  days  and  have  the  fifth: day  to  return  to  their 
^offices  -to  provide  continuity  and  direction.. 


BI-LEVEL 
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This  model  could  approach  the  areas  of  instruction  as  wcil  as 
participant  selection  on  a  bi-level  basis.     That  is  to  say,  research  and 
evaluation  could  be  dealt  with  as  two  distinct  entities  with  specific  instruc- 
tional offerings  for  each,  but  still  employing  the  same  institute  time  frame. 

A  possible  solution  to  the  past  problems  of  heterogeneous  and 
homogeneous  groupings  might  be  using  the  "star"  consultants  with  those 
participants  having  the  greatest  expertise  in  an  area  and  less  well  known 
but  comparably  trained  staff  for  participants  seeking  a  survey  of  the  knowledge 
in  that  area.     The  consultant  would  be  operating  on  an  instructional  level 
maximizing  his  potential  and  yet  would  be  available  to  the  survey  group  om 
occassion  to  provide  them  with  the  credibility  they 'seek. 

FOLLOW-UP 

During  the  academic  year  following  the  summer  institute  a  number  of 
short,  high-intensity  workshops  could  be  held  to  assist  participants  in  trans- 
ferring classroom  experiences  into  the  real  life  situations  confronting  :the 
professional  educator.    These  workshops  or  seminars  would  be  valuable  also  in 
providing  feedback  for  curricula  modifications  for: subsequent  institutes,  the 
development  of  instructional  packages  for  broader  in-service  training  and;^^^^;^'^^ 

hopefully  guarantee  a  utilization,  by  the  participant,  of  the  concepts  has,/: 
learned  in  the:  institute  once  he  returned  to  his  position. 
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